[DEV][TEMPLATE] AnyKernel3 - Easily Mod ROM Ramdisk + Pack Image.gz [Flashable Zip]

Search This thread

osm0sis

Senior Recognized Developer / Contributor
Mar 14, 2012
16,691
40,154
Halifax
GT-i9250
Google Nexus 4
*** NOTE: I will not respond to any requests for support that do not include at least an attached full log and the zip in question; full AK3-debugging.zip debug .tgz preferred (instructions below). ***

AnyKernel3 -- Flashable Zip Template for Kernel Releases with Ramdisk Modifications


"AnyKernel is a template for an update.zip that can apply any kernel to any ROM, regardless of ramdisk." - Koush

The concept of AnyKernel has been around for awhile, (originally by Koushik Dutta/ClockworkMod,) which allowed a device-specific kernel zImage to be flashed over device-specific ROM and use the ramdisk that came with the ROM to reduce the chance of any issues arising from the custom kernel pairing.

The drawback to this was that some kernels require modifications to the ramdisk to enable/set up kernel features, and in the old AnyKernel format there was no way to do this. Enter AnyKernel2.

AnyKernel2 pushed the format even further by allowing kernel developers to modify the underlying ramdisk for kernel feature support easily using a number of included command methods along with properties and variables to customize the installation experience to their kernel. AnyKernel3 adds the power of topjohnwu's magiskboot for wider format support by default.

A script based on Galaxy Nexus (tuna) is included for reference. An example of ramdisk-only changes can be seen in my GN Synapse Injector repo. For an example that also modifies ROM and properly injects init.d support using busybox run-parts and sepolicy-inject see CosmicDan's CosmicTweaks project. For a multi-partition example and an example of how to handle a device which only has a ramdisk when rooted see my N5X/6P BLOD Workaround Injector. Other working AK2/3 examples for more recent devices may be found on eng.stk's blu_spark device repos under Releases.

Please see the linked post here for instructions on working with arm64-only devices (e.g. Pixel 8 series).

Please also see the post here for important notes about the current state of AOSP vendor_boot v4 support and AVBv2 flag options.


Magisk root is automatically detected and retained by patching the new Image.*-dtb as Magisk would!


My development work on my many projects comes out of my free time, so if you enjoy this project or anything else I've done on xda, please consider sponsoring my ongoing work using my GitHub Sponsors profile. For a one-time donation you can hit the donate link from my profile. Thank you for your support!


Source:
https://github.com/osm0sis/AnyKernel3/
Download: https://github.com/osm0sis/AnyKernel3/archive/master.zip


Instructions

1)
Place final kernel build product, e.g. Image.gz-dtb or zImage to name a couple, in the zip root (any separate dt, dtb, recovery_dtbo, dtbo and/or vendor_dlkm should also go here for devices that require custom ones, each will fallback to the original if not included)
2) Place any required ramdisk files in /ramdisk (/vendor_ramdisk for simple multi-partition vendor_boot v3 support) and module files in /modules (with the full path like /modules/system/lib/modules)
3) Place any required patch files (generally partial files which go with AK3 file editing commands) in /patch (/vendor_patch for simple multi-partition vendor_boot v3 support)
4) Modify the anykernel.sh to add your kernel's name, boot partition location, permissions for any added ramdisk files, and use methods for any required ramdisk modifications (optionally, also place banner and/or version files in the root to have these displayed during flash)
5) `zip -r9 UPDATE-AnyKernel3.zip * -x .git -x .github README.md *placeholder`

The LICENSE file must remain in the final zip to comply with licenses for binary redistribution and the license of the AK3 scripts.


If supporting a recovery that forces zip signature verification (like Cyanogen Recovery) then you will need to also sign your zip using the method I describe here:
[DEV][TEMPLATE] Complete Shell Script Flashable Zip Replacement + Signing [SCRIPT]

Not required, but any tweaks you can't hardcode into the source (best practice) should be added with an additional init.tweaks.rc or bootscript.sh to minimize the necessary ramdisk changes. On newer devices Magisk allows these within /overlay.d - see examples.

It is also extremely important to note that for the broadest AK3 compatibility it is always better to modify a ramdisk file rather than replace it.

If running into trouble when flashing an AK3 zip, the suffix -debugging may be added to the zip's filename to enable creation of a debug .tgz of /tmp for later examination while booted or on desktop.


Staying Up-To-Date

Now that you've got a ready zip for your device, you might be wondering how to keep it up-to-date with the latest AnyKernel commits. AnyKernel2 and AnyKernel3 have been painstakingly developed to allow you to just drop in the latest update-binary and tools directory and have everything "just work" for beginners not overly git or script savvy, but the best practice way is as follows:

1) Fork my AnyKernel3 repo on GitHub
2) `git clone https://github.com/<yourname>/AnyKernel3`
3) `git remote add upstream https://github.com/osm0sis/AnyKernel3`
4) `git checkout -b <devicename>`
5) Set it up like your <devicename> zip (i.e. remove any folders you don't use like ramdisk or patch, delete README.md, and add your anykernel.sh and optionally your Image.*-dtb if you want it up there) then commit all those changes
6) `git push --set-upstream origin <devicename>`
7) `git checkout master` then repeat steps 4-6 for any other devices you support

Then you should be able to `git pull upstream master` from your master branch and either merge or cherry-pick the new AK3 commits into your device branches as needed.


Enjoy!
Questions, comments and feedback welcome.



Credits & Thanks: All authors of the included binaries and the tools I used to port them over for their amazing work. koush for the original AnyKernel concept.

Disclaimer: Naturally, you take all the responsibility for what happens to your device when you start messing around with things.
 
Last edited:

osm0sis

Senior Recognized Developer / Contributor
Mar 14, 2012
16,691
40,154
Halifax
GT-i9250
Google Nexus 4
Script Commands Reference

Everything to edit is self-contained in anykernel.sh. A quick-reference for the commands and properties included are as follows.


Properties / Variables

These are some values that will be read during the install process, allowing you to customize your installation, e.g. block= is a shell variable to specify the kernel/boot block partition that the dump_boot command method will copy and unpack.

Code:
kernel.string=KernelName by YourName @ xda-developers
do.devicecheck=1
do.modules=1
do.systemless=1
do.cleanup=1
do.cleanuponabort=0
device.name1=maguro
device.name2=toro
device.name3=toroplus
device.name4=tuna
supported.versions=6.0 - 7.1.2
supported.patchlevels=2019-07 -
supported.vendorpatchlevels=2013-07

block=/dev/block/platform/omap/omap_hsmmc.0/by-name/boot;
is_slot_device=0;
ramdisk_compression=auto;
patch_vbmeta_flag=auto;

do.devicecheck=1 specified requires at least device.name1 to be present. This should match ro.product.device, ro.build.product, ro.product.vendor.device or ro.vendor.product.device from the build.prop files for your device. There is support for as many device.name# properties as needed. You may remove any empty ones that aren't being used.

do.modules=1 will push the .ko contents of the modules directory to the same location relative to root (/) and apply correct permissions. On A/B devices this can only be done to the active slot.

do.systemless=1 (with do.modules=1) will instead push the full contents of the modules directory to create a simple "ak3-helper" Magisk module, allowing developers to effectively replace system files, including .ko files. If the current kernel is changed then the kernel helper module automatically removes itself to prevent conflicts.

do.cleanup=0 will keep the zip from removing its working directory in /tmp/anykernel (by default) - this can be useful if trying to debug in adb shell whether the patches worked correctly.

do.cleanuponabort=0 will keep the zip from removing its working directory in /tmp/anykernel (by default) in case of installation abort.

supported.versions= will match against ro.build.version.release from the current ROM's build.prop. It can be set to a list or range. As a list of one or more entries, e.g. 7.1.2 or 8.1.0, 9 it will look for exact matches, as a range, e.g. 7.1.2 - 9 it will check to make sure the current version falls within those limits. Whitespace optional, and supplied version values should be in the same number format they are in the build.prop value for that Android version.

supported.patchlevels= and supported.vendorpatchlevels= will match against ro.build.version.security_patch and ro.vendor.build.security_patch, respectively, from the current system/vendor build.prop. It can be set as a closed or open-ended range of dates in the format YYYY-MM, whitespace optional, e.g. 2019-04 - 2019-06, 2019-04 - or - 2019-06 where the last two examples show setting a minimum and maximum.

block=auto instead of a direct block filepath enables detection of the device boot partition for use with broad, device non-specific zips. Also accepts any partition filename (from by-name), e.g. boot, recovery, or vendor_boot.

is_slot_device=1 enables detection of the suffix for the active boot partition on slot-based devices and will add this to the end of the supplied block= path. Also accepts auto for use with broad, device non-specific zips.

ramdisk_compression=auto allows automatically repacking the ramdisk with the format detected during unpack. Changing auto to gz, lzo, lzma, xz, bz2, lz4, or lz4-l (for lz4 legacy) instead forces the repack as that format, and using cpio or none will (attempt to) force the repack as uncompressed.

patch_vbmeta_flag=auto allows automatically using the default AVBv2 vbmeta flag on repack, and use the Magisk configuration (Canary 23016+). Set to 0 forces keeping whatever is in the original AVBv2 flags, and set to 1 forces patching the flag (only necessary on few devices).

customdd="<arguments>" may be added to allow specifying additional dd parameters for devices that need to hack their kernel directly into a large partition like mmcblk0, or force use of dd for flashing.

slot_select=active|inactive may be added to allow specifying the target slot. If omitted the default remains active.

no_block_display=1 may be added to disable output of the detected final used partition+slot path for zips which choose to include their own custom output instead.

no_magisk_check=1 may be added to disable detection of Magisk and related kernel/dtb repatching for special zips which don't require that.


Command Methods

Code:
ui_print "<text>" [...]
abort ["<text>" [...]]
contains <string> <substring>
file_getprop <file> <property>

set_perm <owner> <group> <mode> <file> [<file2> ...]
set_perm_recursive <owner> <group> <dir_mode> <file_mode> <dir> [<dir2> ...]

dump_boot
split_boot
unpack_ramdisk

backup_file <file>
restore_file <file>
replace_string <file> <if search string> <original string> <replacement string> <scope>
replace_section <file> <begin search string> <end search string> <replacement string>
remove_section <file> <begin search string> <end search string>
insert_line <file> <if search string> before|after <line match string> <inserted line>
replace_line <file> <line replace string> <replacement line> <scope>
remove_line <file> <line match string> <scope>
prepend_file <file> <if search string> <patch file>
insert_file <file> <if search string> before|after <line match string> <patch file>
append_file <file> <if search string> <patch file>
replace_file <file> <permissions> <patch file>
patch_fstab <fstab file> <mount match name> <fs match type> block|mount|fstype|options|flags <original string> <replacement string>
patch_cmdline <cmdline entry name> <replacement string>
patch_prop <prop file> <prop name> <new prop value>
patch_ueventd <ueventd file> <device node> <permissions> <chown> <chgrp>

repack_ramdisk
flash_boot
flash_generic <partition name>
write_boot

reset_ak [keep]
setup_ak

"if search string" is the string it looks for to decide whether it needs to add the tweak or not, so generally something to indicate the tweak already exists. "cmdline entry name" behaves somewhat like this as a match check for the name of the cmdline entry to be changed/added by the patch_cmdline function, followed by the full entry to replace it. "prop name" also serves as a match check in patch_prop for a property in the given prop file, but is only the prop name as the prop value is specified separately.

Similarly, "line match string" and "line replace string" are the search strings that locate where the modification needs to be made for those commands, "begin search string" and "end search string" are both required to select the first and last lines of the script block to be replaced for replace_section, and "mount match name" and "fs match type" are both required to narrow the patch_fstab command down to the correct entry.

"scope" may be specified as "global" to force all instances of the string/line targeted by replace_string, replace_line or remove_line to be replaced/removed accordingly. Omitted or set to anything else and it will perform the default first-match action.

"before|after" requires you simply specify "before" or "after" for the placement of the inserted line, in relation to "line match string".

"block|mount|fstype|options|flags" requires you specify which part (listed in order) of the fstab entry you want to check and alter.

dump_boot and write_boot are the default method of unpacking/repacking, but for more granular control, or omitting ramdisk changes entirely ("OG AK" mode), these can be separated into split_boot; unpack_ramdisk and repack_ramdisk; flash_boot respectively. flash_generic can be used to flash an image to the corresponding partition. It is automatically included for dtbo and vendor_dlkm in write_boot but can be called separately if using "OG AK" mode or creating a simple partition flashing only zip.

Multi-partition zips can be created by removing the ramdisk and patch folders from the zip and including instead "-files" folders named for the partition (without slot suffix), e.g. boot-files + recovery-files, or kernel-files + ramdisk-files (on some Treble devices). These then contain Image.gz, and ramdisk, patch, etc. subfolders for each partition. To setup for the next partition, simply set block= (without slot suffix) and ramdisk_compression= for the new target partition and use the reset_ak command.

Similarly, multi-slot zips can be created with the normal zip layout for the active (current) slot, then resetting for the inactive slot by setting block= to the partition (without slot suffix) again, slot_select=inactive and ramdisk_compression= to the desired options for the target slot and using the reset_ak keep command, which will retain the patch and any added ramdisk files for the next slot.

backup_file may be used for testing to ensure ramdisk changes are made correctly, transparency for the end-user, or in a ramdisk-only "mod" zip. In the latter case restore_file could also be used to create a "restore" zip to undo the changes, but should be used with caution since the underlying patched files could be changed with ROM/kernel updates.

You may also use ui_print "<text>" to write messages back to the recovery during the modification process, abort "<text>" to abort with optional message, and file_getprop "<file>" "<property>" and contains "<string>" "<substring>" to simplify string testing logic you might want in your script.


Binary Inclusion

The AK3 repo includes current ARM builds of magiskboot, magiskpolicy, lptools_static, httools_static, fec, snapshotupdater_static and busybox by default to keep the basic package small. Builds for other architectures and optional binaries (see below) are available from the latest Magisk zip, or my latest AIK-mobile and Flashlt packages, respectively, here:

https://xdaforums.com/t/tool-androi...kernel-ramdisk-win-android-linux-mac.2073775/ (Android Image Kitchen thread)
https://xdaforums.com/t/tools-zips-scripts-osm0sis-odds-and-ends-multiple-devices-platforms.2239421/ (Odds and Ends thread)

Optional supported binaries which may be placed in /tools to enable built-in expanded functionality are as follows:

mkbootfs - for broken recoveries, or, booted flash support for a script/app via bind mount to /tmp (deprecated/use with caution)
flash_erase, nanddump, nandwrite - MTD block device support for devices where the dd command is not sufficient
dumpimage, mkimage - DENX U-Boot uImage format support
mboot - Intel OSIP Android image format support
unpackelf, mkbootimg - Sony ELF kernel.elf format support, repacking as AOSP standard boot.img for unlocked bootloaders
elftool (with unpackelf) - Sony ELF kernel.elf format support, repacking as ELF for older Sony devices
mkmtkhdr (with unpackelf) - MTK device boot image section headers support for Sony devices
futility + chromeos test keys directory - Google ChromeOS signature support
boot_signer-dexed.jar (deprecated) + avb keys directory - Google Android Verified Boot 1.0 (AVBv1) custom signature support
rkcrc - Rockchip KRNL ramdisk image support

Optionally moving ARM builds to tools/arm and putting x86 builds in tools/x86 will enable architecture detection for use with broad, device non-specific zips.
 
Last edited:

osm0sis

Senior Recognized Developer / Contributor
Mar 14, 2012
16,691
40,154
Halifax
GT-i9250
Google Nexus 4
Thanks guys!

I figured it would be nice to get it out there and also have it as a "Help Desk" thread for kernel devs who have questions about implementation, etc. too. Some devices might require switching it from dd to MTD-Utils, so I can help with that. So on and so forth. :)

Once we get a few devs who know how to use it, it should be pretty easy to help others. I'm looking at you Smitty. ;) No pressure. :p
 
Last edited:

milojoseph

Senior Member
Sep 18, 2012
2,174
721
Brooklyn
I finished my thanks ... but as always a great job.

So wait im confused. ?.. so 1) those that mean I can flash ak kerenl 4.2 with ur any kernel to my 4.4 o_O.
2) those it have to be same kerenl for same phone manufacturer. Meaning can I be stupid enought to flash a nexus 4 kernel in my gnexus?
I understand any kernel cause I have been using smitty so thanks
 

osm0sis

Senior Recognized Developer / Contributor
Mar 14, 2012
16,691
40,154
Halifax
GT-i9250
Google Nexus 4
So wait im confused. ?.. so 1) those that mean I can flash ak kerenl 4.2 with ur any kernel to my 4.4 o_O.
2) those it have to be same kerenl for same phone manufacturer. Meaning can I be stupid enought to flash a nexus 4 kernel in my gnexus?
I understand any kernel cause I have been using smitty so thanks

Haha I wrote "device-specific" in the OP to try and avoid this very confusion. :p

Since I answered this same question earlier tonight in my Odds and Ends thread I'll just paste it here:
so now devs can use kernels from other devices with their roms?

No. That's crazy talk. :laugh:;)

The concept of AnyKernel has been around for awhile, (originally by Koushik Dutta/ClockworkMod,) which allows device-specific kernels to be flashed over device-specific ROMs and use the ramdisk that came with the ROM to reduce the chance of any issues arising from the custom kernel pairing.

The drawback to this is that some kernels require modifications to the ramdisk to enable/set up kernel features, but in the old AnyKernel format there was no way to do this. Until now. :D

AnyKernel 2.0 makes it easy for kernel devs to use a number of simple command methods to automate the process of adding tweaks into a ROM's underlying ramdisk during the flashing process. :good:

Hopefully that helps. Basically exactly what's in the OP since that's where I edited it in from. ;)

The only way I can explain it any further is with the very basics: that kernel boot.img files contain a zImage and a ramdisk. "AnyKernel Classic" just slaps the custom kernel zImage on top of the ROM's untouched default kernel boot.img ramdisk. AnyKernel 2.0 allows kernel devs to also modify the ramdisk to add anything required for kernel features in addition to the usual repacking it with the custom zImage and flashing it.
 
Last edited:

aosp

Inactive Recognized Developer
Aug 12, 2010
2,135
4,997
AnyKernel will work on my phone now ? Thanks for enhancing awesome @osm0sis but DrRamdisk to the rest of you guys ?
 
  • Like
Reactions: osm0sis

osm0sis

Senior Recognized Developer / Contributor
Mar 14, 2012
16,691
40,154
Halifax
GT-i9250
Google Nexus 4
Github updated with my own forked native compiles of mkbootimg+unpackbootimg. ;) :D

This should expand AnyKernel 2.0 device support a lot by using all the available offsets in mkbootimg, as exported by my drastically updated unpackbootimg. :good:
 
Last edited:

Alkemix

Senior Member
Jun 22, 2012
324
227
Italy
Should already? :)

It uses the ROM kernel ramdisk like AnyKernel always has. Your custom kernel dev just needs to use it. Spread the word. :good: ;)

Recently I had used Your method on Cyano11 but boot stopped on "Google".. new Cyano11 (that required ramdisk changes) had just come out and maybe anykernel 2.0 was not ready yet (I had just discovered your brillant work on It ! : Dita incrociate:).

I'll try again ... if I have trouble going to ask you for help ...

I am already spreading the word ;) : Cool:

Thanks man : Good:
 
  • Like
Reactions: osm0sis

Alkemix

Senior Member
Jun 22, 2012
324
227
Italy
What custom kernel were you trying to adapt to AnyKernel so you could flash it on CM?

Two kernels... My custom kernel (from cyanogenmod sources) and recently Fancy kernel (dirty-fancy)... now I want to try Fancy Kernel .. I need of a hibryd ramdisk for best final results and Your project is perfect for It !!! You're a genius !!!

Please, Can You link me Your dirty-V kernel re-pack by Anykernel 2.0 ? So I can follow It as an example. Thanks a lot...
 

osm0sis

Senior Recognized Developer / Contributor
Mar 14, 2012
16,691
40,154
Halifax
GT-i9250
Google Nexus 4
So if I understand you, you're trying to make an AnyKernel 2.0 of Fancy Kernel so that you can flash it on any ROM for your device?

Should be doable. The DirtyV AnyKernel 2.0 is the example posted to the GitHub repo in the OP. Just follow the instructions to make your own anykernel script so that it will add the /sbin/ scripts and other ramdisk modifications (init.d, etc.) that @boype uses, instead of the DirtyV ones.

Good luck!
 
Last edited:
  • Like
Reactions: Alkemix and poondog

Alkemix

Senior Member
Jun 22, 2012
324
227
Italy
So if I understand you, you're trying to make an AnyKernel 2.0 of Fancy Kernel so that you can flash it on any ROM for your device?

Should be doable. The DirtyV AnyKernel 2.0 is the example posted to the GitHub repo in the OP. Just follow the instructions to make your own anykernel script so that it will add the /sbin/ scripts and other ramdisk modifications (init.d, etc.) that @boype uses, instead of the DirtyV ones.

Good luck!

Yes !

osm0sis ? If I want include init.rc original file by "real" ramdisk can I copy It as is into patch folder ?
 

osm0sis

Senior Recognized Developer / Contributor
Mar 14, 2012
16,691
40,154
Halifax
GT-i9250
Google Nexus 4
It would go against the idea of AnyKernel to include the file like that. Remember, everything automatically comes from the original ramdisk, I just give you the ability to alter those files to add tweaks. :good:
 
Last edited:
  • Like
Reactions: Alkemix

Top Liked Posts

  • There are no posts matching your filters.
  • 325
    *** NOTE: I will not respond to any requests for support that do not include at least an attached full log and the zip in question; full AK3-debugging.zip debug .tgz preferred (instructions below). ***

    AnyKernel3 -- Flashable Zip Template for Kernel Releases with Ramdisk Modifications


    "AnyKernel is a template for an update.zip that can apply any kernel to any ROM, regardless of ramdisk." - Koush

    The concept of AnyKernel has been around for awhile, (originally by Koushik Dutta/ClockworkMod,) which allowed a device-specific kernel zImage to be flashed over device-specific ROM and use the ramdisk that came with the ROM to reduce the chance of any issues arising from the custom kernel pairing.

    The drawback to this was that some kernels require modifications to the ramdisk to enable/set up kernel features, and in the old AnyKernel format there was no way to do this. Enter AnyKernel2.

    AnyKernel2 pushed the format even further by allowing kernel developers to modify the underlying ramdisk for kernel feature support easily using a number of included command methods along with properties and variables to customize the installation experience to their kernel. AnyKernel3 adds the power of topjohnwu's magiskboot for wider format support by default.

    A script based on Galaxy Nexus (tuna) is included for reference. An example of ramdisk-only changes can be seen in my GN Synapse Injector repo. For an example that also modifies ROM and properly injects init.d support using busybox run-parts and sepolicy-inject see CosmicDan's CosmicTweaks project. For a multi-partition example and an example of how to handle a device which only has a ramdisk when rooted see my N5X/6P BLOD Workaround Injector. Other working AK2/3 examples for more recent devices may be found on eng.stk's blu_spark device repos under Releases.

    Please see the linked post here for instructions on working with arm64-only devices (e.g. Pixel 8 series).

    Please also see the post here for important notes about the current state of AOSP vendor_boot v4 support and AVBv2 flag options.


    Magisk root is automatically detected and retained by patching the new Image.*-dtb as Magisk would!


    My development work on my many projects comes out of my free time, so if you enjoy this project or anything else I've done on xda, please consider sponsoring my ongoing work using my GitHub Sponsors profile. For a one-time donation you can hit the donate link from my profile. Thank you for your support!


    Source:
    https://github.com/osm0sis/AnyKernel3/
    Download: https://github.com/osm0sis/AnyKernel3/archive/master.zip


    Instructions

    1)
    Place final kernel build product, e.g. Image.gz-dtb or zImage to name a couple, in the zip root (any separate dt, dtb, recovery_dtbo, dtbo and/or vendor_dlkm should also go here for devices that require custom ones, each will fallback to the original if not included)
    2) Place any required ramdisk files in /ramdisk (/vendor_ramdisk for simple multi-partition vendor_boot v3 support) and module files in /modules (with the full path like /modules/system/lib/modules)
    3) Place any required patch files (generally partial files which go with AK3 file editing commands) in /patch (/vendor_patch for simple multi-partition vendor_boot v3 support)
    4) Modify the anykernel.sh to add your kernel's name, boot partition location, permissions for any added ramdisk files, and use methods for any required ramdisk modifications (optionally, also place banner and/or version files in the root to have these displayed during flash)
    5) `zip -r9 UPDATE-AnyKernel3.zip * -x .git -x .github README.md *placeholder`

    The LICENSE file must remain in the final zip to comply with licenses for binary redistribution and the license of the AK3 scripts.


    If supporting a recovery that forces zip signature verification (like Cyanogen Recovery) then you will need to also sign your zip using the method I describe here:
    [DEV][TEMPLATE] Complete Shell Script Flashable Zip Replacement + Signing [SCRIPT]

    Not required, but any tweaks you can't hardcode into the source (best practice) should be added with an additional init.tweaks.rc or bootscript.sh to minimize the necessary ramdisk changes. On newer devices Magisk allows these within /overlay.d - see examples.

    It is also extremely important to note that for the broadest AK3 compatibility it is always better to modify a ramdisk file rather than replace it.

    If running into trouble when flashing an AK3 zip, the suffix -debugging may be added to the zip's filename to enable creation of a debug .tgz of /tmp for later examination while booted or on desktop.


    Staying Up-To-Date

    Now that you've got a ready zip for your device, you might be wondering how to keep it up-to-date with the latest AnyKernel commits. AnyKernel2 and AnyKernel3 have been painstakingly developed to allow you to just drop in the latest update-binary and tools directory and have everything "just work" for beginners not overly git or script savvy, but the best practice way is as follows:

    1) Fork my AnyKernel3 repo on GitHub
    2) `git clone https://github.com/<yourname>/AnyKernel3`
    3) `git remote add upstream https://github.com/osm0sis/AnyKernel3`
    4) `git checkout -b <devicename>`
    5) Set it up like your <devicename> zip (i.e. remove any folders you don't use like ramdisk or patch, delete README.md, and add your anykernel.sh and optionally your Image.*-dtb if you want it up there) then commit all those changes
    6) `git push --set-upstream origin <devicename>`
    7) `git checkout master` then repeat steps 4-6 for any other devices you support

    Then you should be able to `git pull upstream master` from your master branch and either merge or cherry-pick the new AK3 commits into your device branches as needed.


    Enjoy!
    Questions, comments and feedback welcome.



    Credits & Thanks: All authors of the included binaries and the tools I used to port them over for their amazing work. koush for the original AnyKernel concept.

    Disclaimer: Naturally, you take all the responsibility for what happens to your device when you start messing around with things.
    132
    Script Commands Reference

    Everything to edit is self-contained in anykernel.sh. A quick-reference for the commands and properties included are as follows.


    Properties / Variables

    These are some values that will be read during the install process, allowing you to customize your installation, e.g. block= is a shell variable to specify the kernel/boot block partition that the dump_boot command method will copy and unpack.

    Code:
    kernel.string=KernelName by YourName @ xda-developers
    do.devicecheck=1
    do.modules=1
    do.systemless=1
    do.cleanup=1
    do.cleanuponabort=0
    device.name1=maguro
    device.name2=toro
    device.name3=toroplus
    device.name4=tuna
    supported.versions=6.0 - 7.1.2
    supported.patchlevels=2019-07 -
    supported.vendorpatchlevels=2013-07
    
    block=/dev/block/platform/omap/omap_hsmmc.0/by-name/boot;
    is_slot_device=0;
    ramdisk_compression=auto;
    patch_vbmeta_flag=auto;

    do.devicecheck=1 specified requires at least device.name1 to be present. This should match ro.product.device, ro.build.product, ro.product.vendor.device or ro.vendor.product.device from the build.prop files for your device. There is support for as many device.name# properties as needed. You may remove any empty ones that aren't being used.

    do.modules=1 will push the .ko contents of the modules directory to the same location relative to root (/) and apply correct permissions. On A/B devices this can only be done to the active slot.

    do.systemless=1 (with do.modules=1) will instead push the full contents of the modules directory to create a simple "ak3-helper" Magisk module, allowing developers to effectively replace system files, including .ko files. If the current kernel is changed then the kernel helper module automatically removes itself to prevent conflicts.

    do.cleanup=0 will keep the zip from removing its working directory in /tmp/anykernel (by default) - this can be useful if trying to debug in adb shell whether the patches worked correctly.

    do.cleanuponabort=0 will keep the zip from removing its working directory in /tmp/anykernel (by default) in case of installation abort.

    supported.versions= will match against ro.build.version.release from the current ROM's build.prop. It can be set to a list or range. As a list of one or more entries, e.g. 7.1.2 or 8.1.0, 9 it will look for exact matches, as a range, e.g. 7.1.2 - 9 it will check to make sure the current version falls within those limits. Whitespace optional, and supplied version values should be in the same number format they are in the build.prop value for that Android version.

    supported.patchlevels= and supported.vendorpatchlevels= will match against ro.build.version.security_patch and ro.vendor.build.security_patch, respectively, from the current system/vendor build.prop. It can be set as a closed or open-ended range of dates in the format YYYY-MM, whitespace optional, e.g. 2019-04 - 2019-06, 2019-04 - or - 2019-06 where the last two examples show setting a minimum and maximum.

    block=auto instead of a direct block filepath enables detection of the device boot partition for use with broad, device non-specific zips. Also accepts any partition filename (from by-name), e.g. boot, recovery, or vendor_boot.

    is_slot_device=1 enables detection of the suffix for the active boot partition on slot-based devices and will add this to the end of the supplied block= path. Also accepts auto for use with broad, device non-specific zips.

    ramdisk_compression=auto allows automatically repacking the ramdisk with the format detected during unpack. Changing auto to gz, lzo, lzma, xz, bz2, lz4, or lz4-l (for lz4 legacy) instead forces the repack as that format, and using cpio or none will (attempt to) force the repack as uncompressed.

    patch_vbmeta_flag=auto allows automatically using the default AVBv2 vbmeta flag on repack, and use the Magisk configuration (Canary 23016+). Set to 0 forces keeping whatever is in the original AVBv2 flags, and set to 1 forces patching the flag (only necessary on few devices).

    customdd="<arguments>" may be added to allow specifying additional dd parameters for devices that need to hack their kernel directly into a large partition like mmcblk0, or force use of dd for flashing.

    slot_select=active|inactive may be added to allow specifying the target slot. If omitted the default remains active.

    no_block_display=1 may be added to disable output of the detected final used partition+slot path for zips which choose to include their own custom output instead.

    no_magisk_check=1 may be added to disable detection of Magisk and related kernel/dtb repatching for special zips which don't require that.


    Command Methods

    Code:
    ui_print "<text>" [...]
    abort ["<text>" [...]]
    contains <string> <substring>
    file_getprop <file> <property>
    
    set_perm <owner> <group> <mode> <file> [<file2> ...]
    set_perm_recursive <owner> <group> <dir_mode> <file_mode> <dir> [<dir2> ...]
    
    dump_boot
    split_boot
    unpack_ramdisk
    
    backup_file <file>
    restore_file <file>
    replace_string <file> <if search string> <original string> <replacement string> <scope>
    replace_section <file> <begin search string> <end search string> <replacement string>
    remove_section <file> <begin search string> <end search string>
    insert_line <file> <if search string> before|after <line match string> <inserted line>
    replace_line <file> <line replace string> <replacement line> <scope>
    remove_line <file> <line match string> <scope>
    prepend_file <file> <if search string> <patch file>
    insert_file <file> <if search string> before|after <line match string> <patch file>
    append_file <file> <if search string> <patch file>
    replace_file <file> <permissions> <patch file>
    patch_fstab <fstab file> <mount match name> <fs match type> block|mount|fstype|options|flags <original string> <replacement string>
    patch_cmdline <cmdline entry name> <replacement string>
    patch_prop <prop file> <prop name> <new prop value>
    patch_ueventd <ueventd file> <device node> <permissions> <chown> <chgrp>
    
    repack_ramdisk
    flash_boot
    flash_generic <partition name>
    write_boot
    
    reset_ak [keep]
    setup_ak

    "if search string" is the string it looks for to decide whether it needs to add the tweak or not, so generally something to indicate the tweak already exists. "cmdline entry name" behaves somewhat like this as a match check for the name of the cmdline entry to be changed/added by the patch_cmdline function, followed by the full entry to replace it. "prop name" also serves as a match check in patch_prop for a property in the given prop file, but is only the prop name as the prop value is specified separately.

    Similarly, "line match string" and "line replace string" are the search strings that locate where the modification needs to be made for those commands, "begin search string" and "end search string" are both required to select the first and last lines of the script block to be replaced for replace_section, and "mount match name" and "fs match type" are both required to narrow the patch_fstab command down to the correct entry.

    "scope" may be specified as "global" to force all instances of the string/line targeted by replace_string, replace_line or remove_line to be replaced/removed accordingly. Omitted or set to anything else and it will perform the default first-match action.

    "before|after" requires you simply specify "before" or "after" for the placement of the inserted line, in relation to "line match string".

    "block|mount|fstype|options|flags" requires you specify which part (listed in order) of the fstab entry you want to check and alter.

    dump_boot and write_boot are the default method of unpacking/repacking, but for more granular control, or omitting ramdisk changes entirely ("OG AK" mode), these can be separated into split_boot; unpack_ramdisk and repack_ramdisk; flash_boot respectively. flash_generic can be used to flash an image to the corresponding partition. It is automatically included for dtbo and vendor_dlkm in write_boot but can be called separately if using "OG AK" mode or creating a simple partition flashing only zip.

    Multi-partition zips can be created by removing the ramdisk and patch folders from the zip and including instead "-files" folders named for the partition (without slot suffix), e.g. boot-files + recovery-files, or kernel-files + ramdisk-files (on some Treble devices). These then contain Image.gz, and ramdisk, patch, etc. subfolders for each partition. To setup for the next partition, simply set block= (without slot suffix) and ramdisk_compression= for the new target partition and use the reset_ak command.

    Similarly, multi-slot zips can be created with the normal zip layout for the active (current) slot, then resetting for the inactive slot by setting block= to the partition (without slot suffix) again, slot_select=inactive and ramdisk_compression= to the desired options for the target slot and using the reset_ak keep command, which will retain the patch and any added ramdisk files for the next slot.

    backup_file may be used for testing to ensure ramdisk changes are made correctly, transparency for the end-user, or in a ramdisk-only "mod" zip. In the latter case restore_file could also be used to create a "restore" zip to undo the changes, but should be used with caution since the underlying patched files could be changed with ROM/kernel updates.

    You may also use ui_print "<text>" to write messages back to the recovery during the modification process, abort "<text>" to abort with optional message, and file_getprop "<file>" "<property>" and contains "<string>" "<substring>" to simplify string testing logic you might want in your script.


    Binary Inclusion

    The AK3 repo includes current ARM builds of magiskboot, magiskpolicy, lptools_static, httools_static, fec, snapshotupdater_static and busybox by default to keep the basic package small. Builds for other architectures and optional binaries (see below) are available from the latest Magisk zip, or my latest AIK-mobile and Flashlt packages, respectively, here:

    https://xdaforums.com/t/tool-androi...kernel-ramdisk-win-android-linux-mac.2073775/ (Android Image Kitchen thread)
    https://xdaforums.com/t/tools-zips-scripts-osm0sis-odds-and-ends-multiple-devices-platforms.2239421/ (Odds and Ends thread)

    Optional supported binaries which may be placed in /tools to enable built-in expanded functionality are as follows:

    mkbootfs - for broken recoveries, or, booted flash support for a script/app via bind mount to /tmp (deprecated/use with caution)
    flash_erase, nanddump, nandwrite - MTD block device support for devices where the dd command is not sufficient
    dumpimage, mkimage - DENX U-Boot uImage format support
    mboot - Intel OSIP Android image format support
    unpackelf, mkbootimg - Sony ELF kernel.elf format support, repacking as AOSP standard boot.img for unlocked bootloaders
    elftool (with unpackelf) - Sony ELF kernel.elf format support, repacking as ELF for older Sony devices
    mkmtkhdr (with unpackelf) - MTK device boot image section headers support for Sony devices
    futility + chromeos test keys directory - Google ChromeOS signature support
    boot_signer-dexed.jar (deprecated) + avb keys directory - Google Android Verified Boot 1.0 (AVBv1) custom signature support
    rkcrc - Rockchip KRNL ramdisk image support

    Optionally moving ARM builds to tools/arm and putting x86 builds in tools/x86 will enable architecture detection for use with broad, device non-specific zips.
    48
    AnyKernel3 (AK3) Release

    Welcome to AK3: AnyKernel2 + Magisk = AnyKernel3!:
    - abstract out execution directory as $home to fascilitate testing individual scripts (defaults to /tmp/anykernel)
    - general script clean ups for uniformity, readability, speed and simplicity
    - simplify consecutive `ui_print` commands where possible

    Backend only fixes:
    - use busybox umount and mount whenever possible to avoid linker warnings in stderr during system-as-root setup
    - use `busybox --install -s` for bin/* setup
    - consolidate setup to abort earlier if there are issues
    - add backwards compatible symlinks for those who might not have updated their anykernel.sh with ak3-core.sh (to be renamed in subsequent commit)

    Tools only fixes:
    - add magiskboot to replace separate support for AOSP and PXA-AOSP formats, with MTK headers, BLOB, DHTB, SEAndroid and Nook signing
    - add magiskpolicy to allow for easy sepolicy patching
    - remove xz and lz4 since magiskboot supports these (and all other formats except lzop)
    - magiskboot also resolves long-standing request for built-in Bump support (#18)

    AK3 Core only fixes:
    - use full `ui_print` function implementation like the backend
    - add `abort` function to simplify exit on errors
    - fix $patch directory being excluded from `reset_ak`
    - update $ramdisk_compression handling for magiskboot
    - update Sony ELF and related support for initial repack as AOSP by using mkbootimg as plug-and-play optional add-in binary
    - update Sony ELF MTK header support using mkmtkhdr only when ELF is initially unpacked by unpackelf
    - add support for MTK RECOVERY ramdisk header type with Sony ELF
    - simplify U-Boot dumpimage/mkimage and Sony ELF unpackelf/mkbootimg split_img file handling
    - update patch_cmdline to handle both magiskboot as header prop file, and unpackelf or mboot as cmdline.txt
    - use `cp -fp` for `backup_file` and `restore_file` to hopefully better preserve attributes if used outside of the ramdisk
    - refactor setup into `setup_ak` to simplify `reset_ak` trigger to find slot and block again
    - add fallback to busybox for compression formats (i.e. lzop) which magiskboot cannot handle

    IMPORTANT NOTES:
    1) forcing repack with uncompressed cpio ramdisk is not currently supported by magiskboot
    2) forcing repack with lzop ramdisk when previously another format is not currently supported by magiskboot

    https://github.com/osm0sis/AnyKernel3/commit/698682123053cab9e3277c9c9df783a97b3e8958

    tools: rename ak2-core.sh to ak3-core.sh:
    https://github.com/osm0sis/AnyKernel3/commit/cba94d3103ec92380717052a69bb6e37b15ad853

    anykernel: Update for AK3:
    https://github.com/osm0sis/AnyKernel3/commit/8724d881595abd529376cb6fe2205b69a05021dd

    README: Update for AK3:
    https://github.com/osm0sis/AnyKernel3/commit/a10312941ade81955d2217caa0e90dae553bb697

    AK3: fix elftool repack cmdline handling:
    https://github.com/osm0sis/AnyKernel3/commit/c8d087bbad6442546a3f85610e3eaa2d867de021

    AK3: allow workaround for dtbo devices with cat /dev/zero issues:
    https://github.com/osm0sis/AnyKernel3/commit/24032c07328cd2ed9b7c2e57d63d351d52425cf0

    README: updates and touchups for AK3:
    https://github.com/osm0sis/AnyKernel3/commit/5fdf659d01f0fed87b44c3dbc6103d4146b694d5

    AK3: resolve non-zero return code introduced by `cat <img> /dev/zero > <block>`:
    https://github.com/osm0sis/AnyKernel3/commit/97de781a55f09cca631982511789e1ff95badc8f

    Alright! The big day is here. Huge thanks to @topjohnwu for working with me to add the necessary features to magiskboot so I could keep virtually all feature parity with AnyKernel2. Plus the rewrite to magiskboot allowed me to do a thorough code review and includes more format support by default and ensures much better ongoing format support as magiskboot continues to be developed. I wrote this intentionally so that basically all current AK2 zips should be able to be immediately upgradeable using the initial patch, or even just dropping the updated tools/core and backend in.

    Of course pay close attention to merge conflicts for the main AK3 rewrite commit and the anykernel.sh updates, especially since many have added an older version of magiskboot/magiskpolicy unofficially to their zips for root repatching, but note you will need the current included builds since they are a couple commits ahead of even the current Magisk Canary. Going forward let me keep the included ARM magiskboot/magiskpolicy up to date, since I'll be ensuring they work correctly before updating them.

    Thanks also to @eng.stk and @Freak07 for their help testing during the AK3 BETA .patch phase. I also throughly tested most things throughout the rewrite on my OnePlus 6T so things should definitely be all good for OnePlus devices. :D:highfive:

    Edit: Added a couple more commits today to resolve issues on devices where dtbo partition flashing is required. Thanks @MSF Jarvis for reporting the issue, and help testing to find a solution. :)
    18
    Happy New Year!

    Despite me being away on vacation here's a big update to module handling thanks to @nathanchance. If you are currently pushing modules with AK2 you will need to move the files under "modules/" to reflect their destination subfolder to avoid breakage, e.g. "modules/system/lib/modules/", "modules/system/vendor/lib/modules/" and/or "modules/vendor/lib/modules/"; this will be pretty future proof so was worth the while.

    update-binary: Change how we handle modules:
    https://github.com/osm0sis/AnyKernel2/commit/6b11373d4ae91b10ce4745e380ee3d4b440fbde6

    README: Update documentation for modules:
    https://github.com/osm0sis/AnyKernel2/commit/218b5774d26f1ff1c0acd3bec273153f5b529cb2

    example: Update modules folder for new location handling:
    https://github.com/osm0sis/AnyKernel2/commit/a0b6db2784a3cd121a4af768a68ec88fc97c550a
    15
    Thanks guys!

    I figured it would be nice to get it out there and also have it as a "Help Desk" thread for kernel devs who have questions about implementation, etc. too. Some devices might require switching it from dd to MTD-Utils, so I can help with that. So on and so forth. :)

    Once we get a few devs who know how to use it, it should be pretty easy to help others. I'm looking at you Smitty. ;) No pressure. :p