[SCRIPTS] ROM, kernel, anykernel updater and Environment-setup scripts

Search This thread

laufersteppenwolf

Inactive Recognized Developer / Retired Forum Mod
Jul 1, 2012
2,840
6,964
Baden-Württemberg
droideveloper.com
Hey guys

I've been working on some scripts to automate my work for quite some time now and thought, why not share it? ;)
So that's what I'm gonna do now :D

For now, we have three scripts: one to build a ROM, one to compile a kernel, to make an anykernel updater zip and to set up and entire build environment on Arch and Ubuntu-based distros (2nd post). The kernel and anykernel scripts, however, are linked, which means the kernel script automatically executes the anykernel script so as soon as you compile a kernel, you get a recovery flashable zip.

A small note: These scripts were made for the LG Optimus 4x HD, but can easily be modified for any other device.

So, you may ask yourself now 'nice, but what do these script do exactly?' well, that's what I'm gonna tell you now ;)



ROM

The ROM build script can be found HERE
This script is made for CM 11, but can be easily modded to be used with other ROMs. I will also add some more scripts to support more ROMs in the future.

In order to make it compatible with your machine and your device, we have several configurable parameters in the beginning of the script:
Code:
# Configurable parameters
ccache_dir=/home/laufersteppenwolf/ccache/CM11
ccache_log=/home/laufersteppenwolf/ccache/CM11/ccache.log
jobs_sync=30
jobs_build=20
rom=cm
rom_version=11
device_codename=p880
If you want to use a special ccache dir, you can define it using the first parameter. Same for ccache's log.
The "jobs_sync" variable defines how many jobs will be running at the same time while running "repo sync" (---> the -j value)
The "jobs_build" variable defines how many jobs will be used for the make command
"rom" defines the ROM you are trying to compile (used in the lunch command, but also for the zip upload command)
"rom_version" defines the version of the ROM. You can change it to 10.2 or any other version depending on the CM version you are compiling.
Last but not least, the devices codename. In my case it's "p880", but you can just change it to your device (mako, hammerhead, maguro,...) and the lunch as well as the upload command will be changed to your device.

About the upload, if you have a goo.im account and also configured it in your ssh_config, you can just leave it the way it is. However, if you use a different file hoster, please edit the last command to your needs ;)

Furthermore, we have some more features. These, however, don't need to be defined inside the script, but added to the execution command.
You can execute the script with the command:
Code:
. build.sh [-h -n -r -c -ncc -d -j #]
But what do these flags do?
Code:
-h   | --help
shows a help message and exits the script

Code:
-n   | --nosync
skips the "repo sync" command

Code:
-r   | --release
uploads the build when it's done

Code:
-c   | --clean
executes "make clean" instead of "make installclean"

Code:
-ncc | --no_ccache
disables ccache

Code:
-d   | --debug
shows some debug stuff (not really needed for the "normal" user ;) )

Code:
-j #
sets a custom number of jobs for the "make" command


Example:
Code:
. build.sh -c -r
This command syncs the repos, executes "make clean" instead of "make installclean" and uploads the build to the defined file hoster as soon as the build is done.



Kernel

The kernel build script can be found HERE
This script is made for the p880 (aka x3), but can be easily modded to work with your device.

We have 2 configurable parameters in the beginning of the script:
Code:
defconfig=cyanogenmod_x3_defconfig
jobs=32
"defconfig" defines the name of your defconfig to be used
And "jobs" defines the -j value to be used for make

All modules are being copied to ./out/modules/ and the zImage is being copied to ./out/zImage. Which means no more searching for modules inside the various folders, but having everything in one place ;)

If you take a look into the script, the created out dir is after the above command copied to "~/smb/kernel/out". This is where my network drive is mounted. Please change it to a path of your choice.
This folder now needs to have the files from the anykernel folder inside, because the anykernel build script is being executed next. What this script exactly does will be explained later.
When the anykernel updater zip is done, the script cd's back to the place it was initially executed and will then be done.

If you don't want to make an anykernel zip, just remove the commands after
Code:
find -name '*.ko' | xargs -I {} cp {} ./out/modules/
(well, maybe except for the commands that tell you the script it done ;) :p )



Anykernel updater zip

Wanna know what an anykernel updater zip is?
The anykernel updater zip only replaces the actual kernel (zImage) and not the whole boot.img. This means the ramdisk stays the same, which makes the kernel compatible with every ROM and Android version.


The script and it's needed tools can be found HERE

This script creates an anykernel updater zip from a given zImage and modules. The zImage has to be in the same folder as the script itself. The modules have to be in a folder called "modules", which also should be located in the same folder as the script. In the beginning of the script, you can define the name of the zip:
Code:
zipname="WWJB_v009_anykernel.zip"

IMPORTANT:
Please make sure to edit the updater-script so it is compatible with your device! Otherwise you can (hard) brick your device!
I am not responsible for possibly bricked devices!


Another feature of the anykernel updater is, you can mod the ramdisk while flashing (repacking the boot.img). This can be done inside THIS file. By default it creates an unsecured boot image. But you can edit way, way more using this method. For an example what can be done, you might wanna take a look at THIS file ;)

The signapk.jar and the sign keys were taken from the latest CM11 sources.


You have a question, suggestion, bug report? Please feel free to leave a reply in this thread ;) :)



Many thanks go to @GermainZ and @doixanh for helping me fighting with bash ;)
Thanks also go to Koush, for "inventing" the anykernel updater
 
Last edited:

laufersteppenwolf

Inactive Recognized Developer / Retired Forum Mod
Jul 1, 2012
2,840
6,964
Baden-Württemberg
droideveloper.com
Environment setup script

Who is bored of setting up an entire build environment for Android? I certainly am, which is why I wrote this "tiny" script :D

This script sets up a complete build environment, so you can directly start to repo sync a ROM after the script is done. And the best thing is, it is extremely customizable without even having to edit the script itself. All done by appending the appropriate flags to the executing command.
A further big advantage of this script is, that it is compatible with the two most common Linux distros Arch Linux and Ubuntu (also including all distros that are based on these 2, like Xubuntu, Lubuntu,....). This compatibility is achieved by detecting the distro in the beginning of the script and then using the appropriate commands (apt-get or pacman).

The script is located HERE and called "setup.sh"

To run the script, cd to the folder the script is located and enter ". setup.sh" and append all desired flags. If you need help, just append the "-h" or "--help" flag and some help will be shown.

So, you may ask yourself now "That's nice, but what the hell does this script install/set up?" and the answer is quite simple: All you need, and (if desired) even more :D

The "base package" includes:
  • Ccache
  • Java
  • GIT
  • All needed build tools
  • Python 2
  • The Android SDK

Furthermore, you can also choose the "extended package" by appending the flag "-e" or "--extended", which includes:
  • SSH
  • iostat
  • bmon
  • htop
  • geany
  • The dev-host commandline tool by GermainZ
  • udev-rules (on Arch only)

If this still isn't enough for you, you can also add your own packages. This can be done by either editing the script and entering the packages inside the quotes of the following line:
Code:
extra=""			# Add here some extra packages to install
OR
by appending the flag "-s" or "--special" and then listing the packages within quotes, seperated by spaces.
Example:
Code:
. setup.sh -s "package1 package2 package3"


This script is made to perform all actions on its own, however, it will not do so by default. To have it automated, you need to append either "-a" or "--automated". The script will then install the "base package" with all default packages (such as Oracle's JDK 6).
But you can still also append the other flags, like "-e", "-s", "-j",...


Now, let's talk about Java. This script can install 3 Java versions: Oracle's JDK 6, openJDK 6 and openJDK 7. The default version is Oracle's JDK 6. You can change the version by either appending the "-j" or "--java" flag followed by the desired version (1, 2 or 3), or if the -a flag isn't triggered, the script will ask you about it.


What? You don't want to install/change java at all? No problem! :D Just append "--no-java" and Java will get completely skipped.
The same thing also happens to the SDK when appending "--no-sdk".



The script has been tested on (X)ubuntu 13.10 and 14.04 by @nilse and me, and on Arch Linux by @SMillerNL and me. However, I have only tested Oracle's JDK 6, no other java versions but according to the wiki pages, it should work just fine ;)

You have suggestions, feedback, improvements? Shoot! Just let me know and I'll do my best to include it :)

A big thanks goes to my testers @nilse and @SMillerNL who also made some nice suggestions. But also to @GermainZ for his dev-host commandline tool
Furthermore to @eagleeyetom for giving me the idea ;)
 
Last edited:

xboxfanj

Recognized Developer
Apr 24, 2011
7,313
7,797
St. Louis
The anykernel script doesn't work on newer devices with zImage-dtb right?

Doesn't look like it based on the script, but I've had trouble with the M8 DTB so figured I'd ask.
 

laufersteppenwolf

Inactive Recognized Developer / Retired Forum Mod
Jul 1, 2012
2,840
6,964
Baden-Württemberg
droideveloper.com
The anykernel script doesn't work on newer devices with zImage-dtb right?

Doesn't look like it based on the script, but I've had trouble with the M8 DTB so figured I'd ask.

To be honest, I have no idea :D
I have only tested it on the 4x HD myself, but if you use all the same commands as the the ROM build process does (maybe also grab the needed files) it should work. I mean, it does exactly the same thing as when the ROM build packs the boot image, it puts the zImage and ramdisk into one file. Only anykernel does this on the device itself, rather than on your PC.

If you want, we can take a look into it together ;)


Sure, should work with all ROMs, just edit some stuff (like rom_version=11 to rom_version=9 when compiling CM) and it should work just fine
 

Marília de Oliveira

Senior Member
Nov 11, 2012
2,127
2,117
Wonderful city of the sun
www.youtube.com
To be honest, I have no idea :D
I have only tested it on the 4x HD myself, but if you use all the same commands as the the ROM build process does (maybe also grab the needed files) it should work. I mean, it does exactly the same thing as when the ROM build packs the boot image, it puts the zImage and ramdisk into one file. Only anykernel does this on the device itself, rather than on your PC.

If you want, we can take a look into it together ;)



Sure, should work with all ROMs, just edit some stuff (like rom_version=11 to rom_version=9 when compiling CM) and it should work just fine

How do I edit? Can you explain me better .. Thanks
 

laufersteppenwolf

Inactive Recognized Developer / Retired Forum Mod
Jul 1, 2012
2,840
6,964
Baden-Württemberg
droideveloper.com
How do I edit? Can you explain me better .. Thanks

It's quite simple, open the script with an editor (I personally like Geany) and edit the following lines:
Code:
ccache_dir=/home/laufersteppenwolf/ccache/CM11
ccache_log=/home/laufersteppenwolf/ccache/CM11/ccache.log
jobs_sync=30
jobs_build=20
rom=cm
rom_version=11
device_codename=p880

example:
You want to compile CM9 for the Nexus 5 (codename hammerhead) on a dual core CPU and your username (on your PC) is username:
Code:
ccache_dir=/home/[COLOR="Red"]username[/COLOR]/ccache/CM11
ccache_log=/home/[COLOR="red"]username[/COLOR]/ccache/CM11/ccache.log
jobs_sync=[COLOR="red"]5[/COLOR]
jobs_build=[COLOR="red"]3[/COLOR]
rom=cm
rom_version=[COLOR="red"]9[/COLOR]
device_codename=[COLOR="red"]hammerhead[/COLOR]

and you're already done ;) Your script can now compile CM9 for the Nexus 5 ;) :)
 

Marília de Oliveira

Senior Member
Nov 11, 2012
2,127
2,117
Wonderful city of the sun
www.youtube.com
It's quite simple, open the script with an editor (I personally like Geany) and edit the following lines:
Code:
ccache_dir=/home/laufersteppenwolf/ccache/CM11
ccache_log=/home/laufersteppenwolf/ccache/CM11/ccache.log
jobs_sync=30
jobs_build=20
rom=cm
rom_version=11
device_codename=p880

example:
You want to compile CM9 for the Nexus 5 (codename hammerhead) on a dual core CPU and your username (on your PC) is username:
Code:
ccache_dir=/home/[COLOR="Red"]username[/COLOR]/ccache/CM11
ccache_log=/home/[COLOR="red"]username[/COLOR]/ccache/CM11/ccache.log
jobs_sync=[COLOR="red"]5[/COLOR]
jobs_build=[COLOR="red"]3[/COLOR]
rom=cm
rom_version=[COLOR="red"]9[/COLOR]
device_codename=[COLOR="red"]hammerhead[/COLOR]

and you're already done ;) Your script can now compile CM9 for the Nexus 5 ;) :)

I have done everything ... where else do I put the script?
 

laufersteppenwolf

Inactive Recognized Developer / Retired Forum Mod
Jul 1, 2012
2,840
6,964
Baden-Württemberg
droideveloper.com
Alright guys, a big update just found its way to github: a script to set up a complete build environment on Ubuntu and Arch-based distros.

If you want me to add support for another distro, please contact me either via PM or via IRC (#TeamFun or #p880-dev) and give me details on what must be changed (install command, packages, ...)


Also the kernel script got some updates, to abort the script if the kernel didn't compile and to copy the anykernel zip directly to dropbox to share it with your testers ;)

Cheers :)
 
  • Like
Reactions: hallodrix

iurnait

Inactive Recognized Developer
Jun 21, 2012
869
1,152
Southern California
You want to compile CM9 for the Nexus 5 (codename hammerhead) on a dual core CPU and your username (on your PC) is username:
Code:
ccache_dir=/home/[COLOR="Red"]username[/COLOR]/ccache/CM11
ccache_log=/home/[COLOR="red"]username[/COLOR]/ccache/CM11/ccache.log
jobs_sync=[COLOR="red"]5[/COLOR]
jobs_build=[COLOR="red"]3[/COLOR]
rom=cm
rom_version=[COLOR="red"]9[/COLOR]
device_codename=[COLOR="red"]hammerhead[/COLOR]

couldnt you make the script universal by using $HOME?
 
  • Like
Reactions: laufersteppenwolf

workdowg

Senior Member
Apr 17, 2011
1,556
691
Seneca Falls
Wow! Great job and well written... I'll be trying this on my old archserver box that I thought I would have to convert to Ubuntu because I couldn't get all the required packages configured correctly. Looking over your scripts you've touched it all!
 

chasmodo

Senior Member
Dec 28, 2011
12,403
41,133
Novi Sad
Thanks a bunch for the scripts, especially the env-setup one. I'll test it one of these days in Manjaro.

However, env-setup/setup.sh has a small bug: after running the script in LM16 (build environment already set up) it wants to reinstall Java no matter which flag is given.

Also, it doesn't detect installed SDK for some reason.



M86v8Ez.png
 
  • Like
Reactions: laufersteppenwolf

Top Liked Posts

  • There are no posts matching your filters.
  • 47
    Hey guys

    I've been working on some scripts to automate my work for quite some time now and thought, why not share it? ;)
    So that's what I'm gonna do now :D

    For now, we have three scripts: one to build a ROM, one to compile a kernel, to make an anykernel updater zip and to set up and entire build environment on Arch and Ubuntu-based distros (2nd post). The kernel and anykernel scripts, however, are linked, which means the kernel script automatically executes the anykernel script so as soon as you compile a kernel, you get a recovery flashable zip.

    A small note: These scripts were made for the LG Optimus 4x HD, but can easily be modified for any other device.

    So, you may ask yourself now 'nice, but what do these script do exactly?' well, that's what I'm gonna tell you now ;)



    ROM

    The ROM build script can be found HERE
    This script is made for CM 11, but can be easily modded to be used with other ROMs. I will also add some more scripts to support more ROMs in the future.

    In order to make it compatible with your machine and your device, we have several configurable parameters in the beginning of the script:
    Code:
    # Configurable parameters
    ccache_dir=/home/laufersteppenwolf/ccache/CM11
    ccache_log=/home/laufersteppenwolf/ccache/CM11/ccache.log
    jobs_sync=30
    jobs_build=20
    rom=cm
    rom_version=11
    device_codename=p880
    If you want to use a special ccache dir, you can define it using the first parameter. Same for ccache's log.
    The "jobs_sync" variable defines how many jobs will be running at the same time while running "repo sync" (---> the -j value)
    The "jobs_build" variable defines how many jobs will be used for the make command
    "rom" defines the ROM you are trying to compile (used in the lunch command, but also for the zip upload command)
    "rom_version" defines the version of the ROM. You can change it to 10.2 or any other version depending on the CM version you are compiling.
    Last but not least, the devices codename. In my case it's "p880", but you can just change it to your device (mako, hammerhead, maguro,...) and the lunch as well as the upload command will be changed to your device.

    About the upload, if you have a goo.im account and also configured it in your ssh_config, you can just leave it the way it is. However, if you use a different file hoster, please edit the last command to your needs ;)

    Furthermore, we have some more features. These, however, don't need to be defined inside the script, but added to the execution command.
    You can execute the script with the command:
    Code:
    . build.sh [-h -n -r -c -ncc -d -j #]
    But what do these flags do?
    Code:
    -h   | --help
    shows a help message and exits the script

    Code:
    -n   | --nosync
    skips the "repo sync" command

    Code:
    -r   | --release
    uploads the build when it's done

    Code:
    -c   | --clean
    executes "make clean" instead of "make installclean"

    Code:
    -ncc | --no_ccache
    disables ccache

    Code:
    -d   | --debug
    shows some debug stuff (not really needed for the "normal" user ;) )

    Code:
    -j #
    sets a custom number of jobs for the "make" command


    Example:
    Code:
    . build.sh -c -r
    This command syncs the repos, executes "make clean" instead of "make installclean" and uploads the build to the defined file hoster as soon as the build is done.



    Kernel

    The kernel build script can be found HERE
    This script is made for the p880 (aka x3), but can be easily modded to work with your device.

    We have 2 configurable parameters in the beginning of the script:
    Code:
    defconfig=cyanogenmod_x3_defconfig
    jobs=32
    "defconfig" defines the name of your defconfig to be used
    And "jobs" defines the -j value to be used for make

    All modules are being copied to ./out/modules/ and the zImage is being copied to ./out/zImage. Which means no more searching for modules inside the various folders, but having everything in one place ;)

    If you take a look into the script, the created out dir is after the above command copied to "~/smb/kernel/out". This is where my network drive is mounted. Please change it to a path of your choice.
    This folder now needs to have the files from the anykernel folder inside, because the anykernel build script is being executed next. What this script exactly does will be explained later.
    When the anykernel updater zip is done, the script cd's back to the place it was initially executed and will then be done.

    If you don't want to make an anykernel zip, just remove the commands after
    Code:
    find -name '*.ko' | xargs -I {} cp {} ./out/modules/
    (well, maybe except for the commands that tell you the script it done ;) :p )



    Anykernel updater zip

    Wanna know what an anykernel updater zip is?
    The anykernel updater zip only replaces the actual kernel (zImage) and not the whole boot.img. This means the ramdisk stays the same, which makes the kernel compatible with every ROM and Android version.


    The script and it's needed tools can be found HERE

    This script creates an anykernel updater zip from a given zImage and modules. The zImage has to be in the same folder as the script itself. The modules have to be in a folder called "modules", which also should be located in the same folder as the script. In the beginning of the script, you can define the name of the zip:
    Code:
    zipname="WWJB_v009_anykernel.zip"

    IMPORTANT:
    Please make sure to edit the updater-script so it is compatible with your device! Otherwise you can (hard) brick your device!
    I am not responsible for possibly bricked devices!


    Another feature of the anykernel updater is, you can mod the ramdisk while flashing (repacking the boot.img). This can be done inside THIS file. By default it creates an unsecured boot image. But you can edit way, way more using this method. For an example what can be done, you might wanna take a look at THIS file ;)

    The signapk.jar and the sign keys were taken from the latest CM11 sources.


    You have a question, suggestion, bug report? Please feel free to leave a reply in this thread ;) :)



    Many thanks go to @GermainZ and @doixanh for helping me fighting with bash ;)
    Thanks also go to Koush, for "inventing" the anykernel updater
    18
    Environment setup script

    Who is bored of setting up an entire build environment for Android? I certainly am, which is why I wrote this "tiny" script :D

    This script sets up a complete build environment, so you can directly start to repo sync a ROM after the script is done. And the best thing is, it is extremely customizable without even having to edit the script itself. All done by appending the appropriate flags to the executing command.
    A further big advantage of this script is, that it is compatible with the two most common Linux distros Arch Linux and Ubuntu (also including all distros that are based on these 2, like Xubuntu, Lubuntu,....). This compatibility is achieved by detecting the distro in the beginning of the script and then using the appropriate commands (apt-get or pacman).

    The script is located HERE and called "setup.sh"

    To run the script, cd to the folder the script is located and enter ". setup.sh" and append all desired flags. If you need help, just append the "-h" or "--help" flag and some help will be shown.

    So, you may ask yourself now "That's nice, but what the hell does this script install/set up?" and the answer is quite simple: All you need, and (if desired) even more :D

    The "base package" includes:
    • Ccache
    • Java
    • GIT
    • All needed build tools
    • Python 2
    • The Android SDK

    Furthermore, you can also choose the "extended package" by appending the flag "-e" or "--extended", which includes:
    • SSH
    • iostat
    • bmon
    • htop
    • geany
    • The dev-host commandline tool by GermainZ
    • udev-rules (on Arch only)

    If this still isn't enough for you, you can also add your own packages. This can be done by either editing the script and entering the packages inside the quotes of the following line:
    Code:
    extra=""			# Add here some extra packages to install
    OR
    by appending the flag "-s" or "--special" and then listing the packages within quotes, seperated by spaces.
    Example:
    Code:
    . setup.sh -s "package1 package2 package3"


    This script is made to perform all actions on its own, however, it will not do so by default. To have it automated, you need to append either "-a" or "--automated". The script will then install the "base package" with all default packages (such as Oracle's JDK 6).
    But you can still also append the other flags, like "-e", "-s", "-j",...


    Now, let's talk about Java. This script can install 3 Java versions: Oracle's JDK 6, openJDK 6 and openJDK 7. The default version is Oracle's JDK 6. You can change the version by either appending the "-j" or "--java" flag followed by the desired version (1, 2 or 3), or if the -a flag isn't triggered, the script will ask you about it.


    What? You don't want to install/change java at all? No problem! :D Just append "--no-java" and Java will get completely skipped.
    The same thing also happens to the SDK when appending "--no-sdk".



    The script has been tested on (X)ubuntu 13.10 and 14.04 by @nilse and me, and on Arch Linux by @SMillerNL and me. However, I have only tested Oracle's JDK 6, no other java versions but according to the wiki pages, it should work just fine ;)

    You have suggestions, feedback, improvements? Shoot! Just let me know and I'll do my best to include it :)

    A big thanks goes to my testers @nilse and @SMillerNL who also made some nice suggestions. But also to @GermainZ for his dev-host commandline tool
    Furthermore to @eagleeyetom for giving me the idea ;)
    2
    couldnt you make the script universal by using $HOME?

    hmm, good idea :D never thought of that one TBH :eek: :D
    Will add it when I'm back home in the evening ;)

    Thanks a bunch for the scripts, especially the env-setup one. I'll test it one of these days in Manjaro.

    However, env-setup/setup.sh has a small bug: after running the script in LM16 (build environment already set up) it wants to reinstall Java no matter which flag is given.

    Also, it doesn't detect installed SDK for some reason.



    M86v8Ez.png

    About java, you mean because the script asks you which java version you want to have installed? I forgot to disable it when --no-java is given, will add it as well later.
    But it is intented that the script asks you about your desired java version when you choose --needed/-n, as it might be you have openJDK installed, but want oracle's java installed. So if the desired java doesn't match the installed one, the desired one will get installed ;)

    And about the SDK, right now the script only looks for the SDK in its own installation dir, which means if you haven't got your SDK installed in ~/adt-bundle/..., the script will not detect your SDK. I am still thinking of a way to detect the SDK no matter where it got installed, but until now, I didn't come up with a proper solution... ;)


    But thanks for the feedback :) Much appreciated



    But also a big thanks to @eagleeyetom for the portal article :)
    1
    open in notepad or gedit or whatever and edit what u want:p :)

    BTW does -j number of jobs should be for example "-j6" or "-j 6"?

    Sry for the late reply, haven't seen it at all :eek:

    it's "-j x", as it only listens to the things behind the actual flag ;)
    1
    Hi, I just wanted to thank you for the setup-script. It really takes the hassle out of setting up the build-environment.

    I just wanted to add something: I am using Linux Mint which, since it's based on Ubuntu, accepts the exact same commands. However, your script detects the commands to be used based on /etc/issue and cannot detect Linux Mint properly. I changed the script slightly, so if you're interested, I can send it to you. (I basically checked for Linux Mint and set distro to Ubuntu)