[GUIDE + HELP]Rom Making, Stock and Compiled, With Emphasis on Moto MSM8960

Please don't quote this post.
Nobody likes an OPs Wall of Text in the middle of page 3.

One of the best things about Android is the ability to make roms, whether it be from source or stock. A lot of guides go into details about how to get your Linux system set up and running, this one won't. I expect that you already have a running Linux system and have followed one of the many guides around to have all the required software installed.

If you don't have Linux installed, preferably Debian or Debian based (I use Debian Testing) and set up for compiling an Android rom, stop reading this, install Linux, and follow one of the Android guides below Everything in this guide requires Linux, I might add in some Windows tools in the future. Ask for Linux install help in what ever disto you choose's forums, not here.

• This is Google's guide for compiling Android
• This is a guide I found here on XDA.

Required Tools
Attached is a zip containing a lot of tools that I use, and sadly, I've acquired them at different times over the years and I don't remember where I got them all at, so giving proper credit isn't possible. If you developed a tool I used here, please know that you have my thanks and gratitude, and if you let me know, I'll remove it or add proper credit.

Some of the included tools I won't even discuss because they're not relevant to this platfore (like sbf_flash...Linux tool for flashing old school moto firmware to the Bravo, Defy, and other omap3 based moto phones), but they're included because I've added all the things I have installed in my local Android $PATH. They are only 1.9mb extracted, so it's not like they take up a lot of space...anyways...

• Just extract all the tools into $HOME/bin, or any other preferred location in your $PATH
• Ensure they have the proper permissions; run "chmod 755 +x $HOME/bin/*", change $HOME/bin to where ever you installed the tools at
• log out
• log back in

and you're done with the tool installation part (there's also a source /blah/blah command that can be used, but log out and log in is universal). I keep all my sources and tools on their own hard drive, located on /android, so nothing is lost if I ever change Linux distros -- All I have to do is add /android/bin to my $PATH and I'm good to go (once I've copy/pasted the commands in one of the above guides )

Some Linux terms/commands

• $HOME -- The current users home directory. I'm logged in as skeevy420, so my $HOME is /home/skeevy420, $HOME/bin is /home/skeevy420/bin
• $PATH -- all the directoies your system will look for executable files....Google it or check Arch Linux's guides, they're great
• su -- you should really know this one....shame on you if you don't
• sudo -i -- similar to sudo, only it opens an interactive sudo shell where every command is done as sudo

Converting a Fastboot into a flashable zip
This is pretty straight forward and only requires dumpperms.pl and simg2img.py (in the zip you should have already installed)
To make things easier all the terminal commands are colored blue

First Step -- Converting system.img into a mountable format

• Download and Extract a fastboot zip file
• Open a terminal in the folder you extracted the zip to
• run "simg2img.py ./system.img ./system.img.raw" (it's system.img.ext4, not system.img, if you're converting an ICS fastboot image)

Second Step -- Mount the converted image

• Obtain a root shell, use "sudo -i" or "su" depending on how your system handles regular users accessing superuser privileges ***$PATH HINT***
• Every single command in every step done after this will be as su/sudo -i
• cd back to the folder with the converted image, if necessary
• mkdir -p /mnt/tmp/zip/system
• mount -o loop system.img.raw /mnt/tmp/zip/system

Thrid Step -- Get file permissions

• dumpperms.pl /mnt/tmp/zip/system > /mnt/tmp/perms

Fourth Step -- Add in META-INF (this is from Gummy Rom June 02, 2014)

• Download the attached META-INF.zip
• unzip META-INF.zip -d /mnt/tmp/zip

Fifth Step -- Modify Updater Script

• Open /mnt/tmp/perms with a text editor
• Select everything and copy it
• Open /mnt/tmp/zip/META-INF/com/google/android/updater-script with a text editor
• Select the marked line, highlighted blue in the template below
• Paste in the permissions
• Save it as /mnt/tmp/zip/META-INF/com/google/android/updater-script

Sixth Step -- Add in kernel/boot.img

• cd into extracted fastboot folder if necessary
• cp boot.img /mnt/tmp/zip

Seventh Step -- Pack it all up

• zip -r /mnt/tmp/zip/rom.zip /mnt/tmp/zip

Eight Step -- Flash it

• You should really know how to get a rom on your phone at this point in the game....
• If you don't, we're all gonna laugh at you.

Updater Script Template

Amended Hints

• ***$PATH HINT*** Make sure root/sudo has access to the primary users $HOME/bin or where every you installed the tools to, It's that or you'll have to figure out your own workarounds and do a lot of jumping between user and su.

THIS IS A WIP, I'LL BE ADDING MORE THINGS IN THE FUTURE. I'M HUNGRY AND I WANT A BREAK FROM TYPING

Please don't quote this post either. Thank you.

Introduction
One of the reasons people download rom sources is to just compile them, but the nostalgia from that wears of pretty fast and you're left thinking -- What Next??? There's plenty of things that can be done, you can add in things like Archidroid's Optimizations, take an AOKP feature and port it to Gummy, you could optimize the build with Linaro or a different GCC version, you might want to try your hand at compiling an optimized toolchain to optimize the rom's compile even further, you could do bug fixes and submit them to CM or your rom of choice's developer..the possibilities are endless, but where to start? How to start?

The example below is based on adding Archidroid's Optimizations to PAC and will be on my GitHub page for anyone to see and use. I'm using Archidroid's Optimizations because they're well documented, they don't cleanly pick with PAC, and they're a neat thing that can be done to nearly any rom -- all that makes it a good candidate for a basic first source based mod tutorial. I'll also be adding in Arrrghhh's kernel because why not.

First thing you need to do is get a Github account.
You're gonna want this. It's free and it gives you a place push your modified sources to; so when you sync, you get your changes. It also serves as a good backup for the projects you're working on. I'll go into remotes and pushing when the time comes.


Now, Archidroid and PAC...where to start?
I'm gonna assume that you've already synced PAC's sources as well as initialized a PAC build to create roomservice.xml and sync the device sources. If you haven't synced yet, but you already have, say, CM or AOSP synced, you can add --reference=/path/to/your/CM/sources and your PAC sync will use your local CM files if they're the same as what PAC needs. --reference is one of my favorite repo commands.

Well, you start by forking all the projects PAC has that are modified by Archidroid's Optimizations, which are android_build, android_art, & for PAC, android_vendor_pac to incorporate it properly into PAC's build script. See, PAC's build script already allows defining the GCC O Level which is a big part of what Archidroid's Optimizations target, but if done right, they can be integrated in a way that allows defining Os, O2, O3, Oa (O Archidroid), and Of (his Ofast commit that he's testing). I might even get around give PAC's build script a way to define GCC version and Google or Linaro GCC...

You're gonna want to open PAC's manifest, located here, and determine from where you fork the 3 projects. You'll see that android_art is from CM, so fork it from Cyanogenmod. To Fork, sign in to Github, go to where the project is located, and click the fork button in the upper right hand corner...simple. android_build and android_vendor_pac are PAC repos, so fork them from PAC.

Thought you were done forking? Nope, you have one more to fork IF you plan on submitting this to PAC's Gerrit like I plan on doing. That fork is PAC-man/pacman because Archidroids GCC repos and (a forked) android_art will have to be added to PAC's manifest.

I'm also going to go into detail and write the guide from the perspective view of using a local manifest, removals, and whatnot -- IMHO, Forking and updating manifests is better if you plan on contributing to a project/submit for Gerrit review, local manifests are better for personal projects, testing, and preparing to do a proper, correct submission to Gerrit review.


Create Local Manifests
Probably the easiest part of AO is adding GCC 4.8. This is done by simply making a file named (pretty much anything except roomservice.xml) linaro.xml. It'll be a standard repo xml file that looks like this


The only problem with that is, for us, Linaro GCC 4.8 built roms seem a bit more error prone, but we can't just add in Linaro GCC 4.7 like above because we'll get sync errors since PAC already has Google GCC 4.7...so we'll add some project removal lines in the xml...sources in local manifests take precedent.


NOTE: The remote lines aren't needed here. They're already in PAC's manifest and I only included them above to get you thinking about your remotes when you're setting up a manifest. If you wanna compile PAC following this guides instructions, before you click save after you copy and paste the above xml, delete lines 3 to 11, making the new line the be the Removals line. You should also check the remotes before you use this with other roms, because this is pretty rom independent; the removal lines may not be necessary on some roms.

Now that we have an xml file that handles our toolchain, I'll break it down and explain where the different things come from.

The first thing you'll notice is the remotes. In the xml files, the remote is the website or location you're getting your sources from. They can also be used to use less characters in the xml -- I could have added a remote named Archi with fetch="https://github.com/JustArchi", then I'd change the remote to Archi and the name to just "Linaro". While the 4 repos here isn't that big of a deal, imagine having 300 name="CyanogenMod/blah" projects, name="blah" remote="cm" seems like a better idea doesn't it?.

Below the remotes are the removals, and they're determined by the name= part of a project line of your main manifest.xml or roomservice.xml. Adding in more removals is simply copy/pasting the name= line of a project and having it formatted like you see above.

The Adds....those are just your standard lines of a manifest containing the stuff you want to add to your project or use now that the crap you don't want to use is removed.


Side Project -- Adding in a Arrrghhh's Custom Kernel to the Local Manifest
This part here is generic and geared towards CyanogenMod. I won't be using this with the rest of the Archidroid tutorial.
Simple two line mod


Adding Arrrghhh's Kernel to PAC Vendor
While the Dev Team's kernel is nice and stable, it just doesn't have all that many tweaks. That's where Arrrghh's kernel comes into play. It's the Dev Team's kernel with both higher and lower frequencies to use, more governors and schedulers, adjustable voltage table, and more.

Before I go on, there are two programs I use that help with merge conflicts, and those are Meld and Git-Cola. I'll be using them below, but if you have a preferred way of dealing with merge conflicts, then by all means use that (I like SmartGit HG too). For most Debian/Ubuntu distros, they can be installed with the following terminal command.


IMPORTANT: I'll be using my own personal repos and directories below. I'll try to point them out, but for general note, /android/pac is my sources' root directory and remotes created named skeevy are mine, you need to create them with a different name and have the URL set to your github account, not mine.

Now that that's out of the way, fire up the terminal and go to your sources root directory, for me it's cd /android/pac, and do a repo sync. Once the sources update, we start.


Fire up your text editor, I use gedit, and use that to open up our pac device dependencies file

And modify the kernel part so it looks like this


Save it. Gedit, by default, saves a backup file in the same directory as the file you saved, make sure you delete that file if it did. And back to the terminal.


Now that Arrrghhhs kernel has been added to PAC, we'll move on to adding Archidroid to android_art....just kidding, we gotta play with removals and local manifests again...remove PAC's vendor and use your forked one.


Android Art
Finally, actually doing something related to Archidroid's Optimizations. Yay.

JUST A TEMP SAVE. I'M STILL WORKING ON THIS
Art -- Remote
Art -- Branch
Art -- Pick
Art -- Push

Build -- Remote
Build -- Branch
Build -- Pick
Build -- Fix\Build Script Integration
Build -- Push

Vendor -- Almost same as above
Vendor -- Archidroid into Pac Build Script
Vendor -- Add Arrrghhh kernel

Pac Compile 0a
Pac Compile 0f
This part of the guide will take me 5 or 6 hours....but you'll get some cool builds out of it

Submitting to PAC's Gerrit if builds are successful, doesn't actually have to boot or work...I'll explain...

Wait a for anything above's upstream to be updated, and an update my forked stuff with master guide.

***BRANCH NOTE*** pac-4.4 (the branch name) can be whatever you want it to be, but if it's something you'll submit for a gerrit review, use the branch the project you're committing to is using. There's also using a temp branch then cherry-picking in the changes when your done...but we're not going into gerrit strategies here.

and a third just in case

EDIT: Feel free to post rom tweaking and compile questions and advise while I'm finishing the guide up. There's quite a bit of content and I'd like to make sure it's correct and working before posting.

So I've done some testing with the Linaro toolchain and the HD. General consensus (and source) seems to indicate the S4 Plus in the device is closer to the Cortex-A9 than the Cortex-A15 but has some extra goodies that "kind of make up for it".

Testing against both the A9 and A15 optimized toolchains, the A9 feels "better" but the A15 works just fine too. Main problem is the razrqcom source in CM11 defaults to A15 so there lies a problem. I will probably have to bite the bullet and compile my own optimized A9 toolchain with VFPv4 (vs VFPv3) and see how it turns out.

Either way, the performance benefit is clearly there. Benchmarks somewhat reflect it but "real world use" it shines vs non-optimized. Once I'm happy, I'll post my findings so others can try their hand at it.

EDIT: I'll make two builds available and update this post. Both are identical except for the toolchain + arch related opts applied. One is Cortex-A15 tuned and one is Cortex-A9 tuned. You decide from there. Keep in mind I'm still using the default GCC (4.7) with no special optimizations applied to it other than for ARM (under build/); everything else is stock and I'm not using my own toolchain yet.

Here we go: http://www.filedropper.com/cm-11-20140630-unofficial-motomsm8960

This is still based on 6-27 source. Not quite where I want it to be but I need to make diffs of my changes before I repo sync now. ART is working. Seems plenty fast/smooth once it settles. I'll be working on my diffs for a while and then producing a newer, hopefully faster build next build. This build is mostly -O3 now but some areas need some tweaking.

EDIT: Sorry Z, no GDrive links yet