[CLOSED][Guide] How-to Automate Your ROM Build Process Using Jenkins - Setup Nightlies

Special thanks to @chasmodo for all his support, as well as his wonderful How-To build CM thread located HERE.

I. Donations and Support...

Feel free to post any question here, as well as provide your own scripts. As far as I'm concerned, there are no stupid questions; you are, however, required to search before posting.

Please do not donate to me, donate to the Testicular Cancer Society (Click Here).

II. Introduction...

This guide will direct you on how to setup your android build environment to automate your builds using Jenkins and various shell/bash scripts. As this was initially drafted, it was targeted towards CyanogenMod 10.2, but I am quite confident it can easily be easily modified to work with anything you choose, including LP builds.

This can be a useful process if you want to setup nightly/weekly builds, or if you just want to have a easy-to-use web interface to run your builds without entering the shell.

This guide assumes you have an active build environment (preferably Ubuntu-based), and general linux knowledge.

III. Initializing your build environment...

You will first have to have your build environment setup, and you should already be able to successfully build CM, AOSP, AOKP, or other variants, before moving forward. Furthermore, you are expected to have simple scripting knowledge, or you will not be able to do this. If you need assistance with setting up an android build environment (which is not the focus of this thread), you can get started by Clicking Here, or by looking at @chasmodo's thread by Clicking Here.

It is important to note that your Jenkins home directory is at "/var/lib/jenkins" (not /home/jenkins). As such, to get your build directory in the proper location, you will login normally to your shell, and "su jenkins" (you may have to use su policy, if you don't know how to do this, you may just want to login as root). Do Not login to the system-jenkins-user shell by changing the jenkins user password (I am referring to the System User named jenkins, not your jenkins web-interface). Once logged in as jenkins, do a "cd ~", create your directory (e.g., "cm10.2"), and repo init there (e.g., in /var/lib/jenkins/cm10.2 directory).

IV. Installing Jenkins and Apache Daemons...

Once your build environment has been initialized, you will have to get Jenkins/Hudson setup and running. You can read more about Jenkins software by Clicking Here.

Further, you will need to install and setup apache web server in order to access the Jenkins web interface. Click Here for more information on apache.

The jenkins install process can easily be setup by following the guide Located Here.

After you have successfully installed Jenkins and apache, you may proceed to setting up your automated builds.

V. Setting Up Jenkins Security...

Be sure to setup a Jenkins user account, or a remote user can get your setup and screw up your machine.
Follow the article here:
https://wiki.jenkins-ci.org/display/JENKINS/Standard+Security+Setup

Continued in Post #2

V. Configuring your first build...

.... A. Creating the new build...

Once you have setup your jenkins server (by going to http://localhost:8080, or port 80 if you set it that way), log in with the username/password you configured.

After you have logged in, click "New Job" in the left toolbar.

Screenshot:

Then, enter the "Job name". Please Note to NOT use spaces or slashes in the name, as it causes build issues. If you need a space, use an underscore ("_"). Check the radio-box "Build a free-style software project", and hit "OK".

Screenshot:

Now, continue to the procedure listed directly below (i.e., in section "B", "How to have Jenkins call your local scripts...")

.... B. How to have Jenkins call your local scripts...

As opposed to attempting to have Jenkins' propriety interface try to run your build process directly, it is best to have it call out for your individualized shell/bash scripts.

To do so, under the "Execute Shell" Field in the "Build" section, you will call out your script by entering the following code:

Screenshot:


Once your code has been input, the next step is to create your script in your build directory. In this instance, the script is called "buildscriptclobber.sh" and the directory is "~/cm10.2" (~ is home). It is important to remember that your home directory is the Jenkins user home directory, which gets installed at /var/lib/jenkins (NOT /home/jenkins).

As above mentioned, you will login normally to your shell, and "su jenkins". Once logged in as jenkins, do a "cd ~/FOLDER" (e.g., "~/cm10.2"). Then, use your favorite text editor (e.g., pico, installed by sudo apt-get install pico), and enter your script.

A good script I wrote will make clobber and repo sync every time. It is as follows:

You may notice above that I call out the folder ("cd"), and set the "PATH" multiple times, especially when its already set in your ".bashrc". This is because the script will give errors if you do not do so, because Jenkins won't consistently read your ".bashrc" or other external configurations. It is important to always do this type of thing when shell scripting with Jenkins. The "time" command may be omitted, because Jenkins log will show the total build time throughout, so its partially redundant, but I like it in there.

Once the script has been setup, do not forget to "chmod a+x" or it will not run.

VI. Automating your first build...

WIP SETTING UP H H H TRIGGER

VII. Configuring and automating additional builds...

Follow the same steps as above, just be sure the Cron Triggers are set to not overlap. For Example, if you have setup H H H(0-5) for one, and you know your builds may take up to two hours, then setup H H H(2-7) for the next.

Continued in Post #3

VIII. Scripting your builds in general...

.... A. Use individual shell scripts for each process...

The beauty of having a Jenkins environment is your scripting capabilities. There are many possibilities, and many plugins available within the Jenkins interface. I, however, prefer just executing Shell Scripts, since you will have more control over what you are doing via bash scripting. If you are a bash scripting novice, you can Click Here to view a nice guide.

When scripting, realize that every change you make must be changed in your other scripts. For example, if I use the change filename script below, obviously I will need to modify my upload script to reflect the changed filename. I think of it in a way that "one thing breaks another". This is why its great to have an independent script for every process you do, so that only one file requires modification if changes are made.

.... B. How to set your scripts to run after the initial compile process has completed...

In order to run ANY OTHER PROCESS after the initial build script is complete, you must run another script after build. To do this, you will use the "Add build step" feature to "Execute Shell", located in the "Build" section. As such, for ANY new script you want to run after the build script, you must add another build step. Consequently, every single script below requires a new build script.

Screenshot:


Like above when you setup your initial build script, you will call a script out. This can be done the same way, using:

Screenshot:


Continued in Post #4

IX. Specific post-build scripts...

.... A. Automating Filename Changes, and/or Moving The Build to a Different Directory Automatically...

This essentially moves the build .zip to the same folder, but renames it to the name you choose. Thus, when it uploads, you don't have to manually rename the ROM Firmware file.


If you want to add the date to the filename, you can add the following string within the filename:


You are free to modify the ordering of %Y%m%d as you see fit.

Therefore, if I wanted my build to say "SGT7-DATE-n7000-NIGHTLY.zip", I would use the following as the filename:


As such, you're final code would be:


Alternatively, if you want to move to a local directory, such as your public ftp folder, you can use the following code instead, adding the date string above if you want:



.... B. Automating uploads to "goo.im" or any other FTP/SFTP server...

First, you will have to install yafc in ubuntu. This is done via "sudo apt-get install yafc".

Then, create a script using the following code:

.... C. Automating build.prop modifications...

MARKER

.... D. Automating changelogs...

MARKER

Continued in Post #5

X. Troubleshooting...

Here are some common solutions to your issues.

• Reboot your system ("reboot -n") and try again. You may need a daily, or even weekly reboot depending on your RAM, software causing memory leaks, etc. You can setup a CRONJOB (Click Here for more info) to do a reboot daily. If you choose to setup a job in the crontab, be ensure it doesn't conflict with the timing of your Jenkins cron builds.
• If you keep getting java out of memory errors, you probably need more RAM.
• Other errors may be caused by your CCACHE being set to low. If you continue having issues caused by ccache, even when its high, you may want to install a global ccache in /usr/bin or the like.
• Did you remember to set permissions on your scripts ("chmod a+x")
• need to reset your username? See this post: http://xdaforums.com/showthread.php?p=51248853
• having ota jenkins.zip NOT FOUND ERROR (or no such file or directory)? See this post: http://xdaforums.com/showthread.php?p=48454597; see also this post: http://xdaforums.com/showthread.php?p=55954963

MikeyCriggz said:

Subscribed!

Sent from my Nexus 4 using Tapatalk 4

Click to expand...

Click to collapse

I have lots of great scripts and stuff I will be posting! just have to juggle my real job with throwing in some time every day to build this up...

chasmodo said:

Well done, mate. This is something that hasn't been covered so far methinks. :good:

Click to expand...

Click to collapse

Thanks buddy. I wanted to PM you a question, but you don't accept PMs...
I'll G+ you later on.

dschense said:

with my new root - 16GB RAM - everything works like it should thread subscribed!

is there any way to make the build faster? yes, using ccache.. but are there some more ways to get everything out of the box ?

Click to expand...

Click to collapse

What type of hard drive do you have?
How long do your Jenkins builds take to run?

The first run always fails (flaw in Jenkins) I need to add that to the guide.
The second run always takes extra long
The third run should be quicker...
My builds take about 30-45 minutes including repo sync everything

If you have a Solid State hard drive you can have builds going much quicker....

@dschense
I'm confused as to what your Jenkins shell command is...
Also are you calling another script within a script? Or do you have two separate Jenkins execute shells?
Does that question make sense?

Also if you went into that particular build directory logged in via su jenkins, can you compile it manually without error?

your setup should look like this:


is that what you have?

You should Not be calling another script from inside your jenkins script. that will cause issues. you need to do them incrementally in chronological order and do a new execute shell for each script.

Also the "./" command won't work inside a script, or in a jenkins command

so your "./rom-build.sh i9300 clean" should be a whole new execute shell
and instead of "./" you need to use the "source" command like I have above

Also can you post the link of your error? I know you PM'd Me maybe u can paste that here

Sent from my GT-N7000 using Tapatalk 2

dschense said:

I think my biggest problem is how I can do a right shell script for building. If I build ParanoidAndroid from source manually I use the command. /rom-build.sh i9300... But I don't know exactly what commands to use for just using the shell window in Jenkins without using the script by ParanoidAndroid.. Any help?

Kaaaaaaaaatsching.... Tapatalked!

Click to expand...

Click to collapse

I'm trying to help but you don't respond to my questions.

Can you (1) post the PA build script and (2) post exactly what is in your execute shell

Sent from my GT-N7000 using Tapatalk 2