[R&D] MYTHBUSTER Optimized compiler toolchains

---

There are several different cross-compiler toolchains available for building the Android Linux kernel. There is the offical TC released by Google as part of the Android NDK (http://developer.android.com/sdk/ndk/index.html) as well as several TCs released by third parties which claim to be more optimized for the target platform. While the most popular of these third party TC is the one from CodeSourcerey (https://sourcery.mentor.com/sgpp/lit...al/release1803), Linaro also offers one TC very popular by developers of custom kernels (http://www.linaro.org/downloads/) and there also is the Mjolnir TC by redstar3894 and intersectRaven (https://github.com/redstar3894/Manif...er/README.mkdn).

I have read of lot of praise for these optimized TC from different kernel developers that all claim that these do much better than the offcial Google TC, however to my knowledge up until now nobody actually took the effort of investigating the effect of the TC and most devs simply assume that a TC marketed as optimized by their creators is actually performing better.

So to investigate the performance of the different TC, I compiled a kernel (GLaDOS kernel for ICS) with the following four different TC:

1. Offical Google arm-linux-androideabi-4.4.3 (part of android-ndk-r7)
2. CodeSourcerey arm-2011.03-41-arm-none-linux-gnueabi
3. Linaro android-toolchain-eabi-linaro-4.6-2011.11-4-2011-11-15_12-22-49-linux-x86
4. Mjolnir arm-eabi-4.6-mjolnir-20111006

Also while I was at it, I investigated the effect of the compiler flags '-fgcse-after-reload' and '-ftree-vectorize' (see https://github.com/Ezekeel/GLaDOS-ne...f32ee#comments) by compiling a version with the CS TC which did not include these flags.


I performed the following two tests for the kernels:

Test I: Measured the bootup time including a rebuild of the Dalvik-cache after a wipe (2 times).
Test II: Performed a benchmark with AnTuTu Benchmark v2.4.3 including CPU/memory, 2D and 3D graphics (3 times).


Results:

Test I:

Google		1:34	1:34
CS		1:34	1:34
CS-flags	1:34	1:34
Linaro		1:34	1:34
Mjolnir		1:34	1:34

Test II:

Google TC:



CodeSourcerey TC:



CodeSourcerey TC without flags:



Linaro:



Mjolnir:



Average:

Google		2715
CS		2725
CS-flags	2718
Linaro		2718
Mjolnir		2716

As one can clearly see from the results, the so-called optimized TCs perform not better than the official TC released by Google in any measurable way. So the improvements that some kernel devs have felt cannot be backed up by these data and is most likely wishful thinking. Also note that the compiler flags '-fgcse-after-reload' and '-ftree-vectorize' do not have to seem any adverse effects regarding the performance of the kernel.




Update:

In case anyone wants to try some benchmark fun for himself, here are the kernels (GLaDOS kernel for ICS).

Google: http://www.multiupload.com/QK5T6SHUFC
CS: http://www.multiupload.com/25021GRFT0
CS-flags: http://www.multiupload.com/98TCZOKLWA
Linaro: http://www.multiupload.com/XZJN2J9DOE
Mjolnir: http://www.multiupload.com/QK4I6R8DGR

First! How do you think of this stuff?!

I know I'm spamming but still. :P

Sent from my Nexus S using XDA App

Let me preface this by saying i haven't compiled since i was a teenager (distant memory) and only built simple msdos apps. My question is, when you compile for Android, does it produce machine code, or something else usable by the VM? Is the kernel also machine code, or something else suitable for the boot loader?

Sent from my SNES

Ezekeel, thank you for taking the time to invastigate on this topic. This thread should be moved to the general "Android Software Development" forum to inform the other devs about this aswell.

O.T.: You should really think about a paypal donation button. I know about your reservation about taking money, but you might make a statement about only taking donations for already made development. This way you are free to decide whether you stop working on something or not and users can't complain about abandoned projects they have donated to.

Awesome, awesome post. I've always figured the TC made little difference to the kernel.

What would be interesting, though, would be to see the effect on the whole OS - in particular, Skia, audio processing, the brwser, etc - all the C/C++ stuff. There's not only a lot more of that code, but there's also a lot more floating point and that could maybe see some useful gains.

You can clearly see CS is the winner here!

Definitely nothing worth mentioning in terms of gains, if any. On 100 runs I bet they'd all get the same average.

Nevertheless, could it be that one gives better results when meeting specific criteria? Thinking mostly about the compiling hardware than the device it's being compiled for. Maybe older/newer/different hardware?

Originally Posted by FloHimself

O.T.: You should really think about a paypal donation button. I know about your reservation about taking money, but you might make a statement about only taking donations for already made development. This way you are free to decide whether you stop working on something or not and users can't complain about abandoned projects they have donated to.

Seconded.

Sent from my Nexus S using XDA App

This is awesome... I wish more of these threads existed!

Any thought on how these have to do with battery life?

LoL, great research Ez!!

Wherever, can't wait for new mods!

Originally Posted by bedalus

Let me preface this by saying i haven't compiled since i was a teenager (distant memory) and only built simple msdos apps. My question is, when you compile for Android, does it produce machine code, or something else usable by the VM? Is the kernel also machine code, or something else suitable for the boot loader?

Sent from my SNES

Android apps are written in Java and run of the Dalvik VM. The kernel is written in C and runs directly on the hardware.

Originally Posted by loudaccord

This is awesome... I wish more of these threads existed!

Any thought on how these have to do with battery life?

Since the performance is virtually the same, I do not expect the battery runtime to be any different either.


Franco has suggested to try the Mjolnir TC with a different set of compiler flags that does enable the hardware support for floating-point math (http://pastie.org/2951626).

Results:

Test I:

Google		1:34	1:34
CS		1:34	1:34
CS-flags	1:34	1:34
Linaro		1:34	1:34
Mjolnir		1:34	1:34
Mjolnir-float	1:34	1:34

Test II:



Average:

Google		2715
CS		2725
CS-flags	2718
Linaro		2718
Mjolnir		2716
Mjolnir-float	2720

As one can see enabling hardware support for floating-point math in the Mjolnir TC does not yield any measurable improvements in performance. I guess the reason is the use of floating-point math in the kernel is avoided at all costs, because for most platforms only a very inefficient software emulation is available and so kernel devs have learned to get along with only integers or if some decimal number are really necessary by using fixed-point math.


GLaDOS ICS kernel with Mjolnir TC and floating-point support: http://www.multiupload.com/AM6COIPI6Z