[WIP] Compile Android (5.x) with Qualcomm LLVM 3.7.1 for Qualcomm CPU

Search This thread

MusterMaxMueller

Senior Member
Mar 12, 2014
224
191
Use Qualcomm Snapdragon LLVM Compiler 3.7.1 (https://developer.qualcomm.com/mobi...-performance/snapdragon-llvm-compiler-android) for Android Lollipop 5.x for devices with Qualcomm CPU

As far as I know nobody has done that yet. Except me :)

This changes the toolchain for CLANG from the default one which comes with Android (aCLANG) to the Qualcomm Snapdragon LLVM Compiler 3.7.1 (qCLANG) mentioned above. The toolchain will be used for almost all places where the default Android CLANG (aCLANG) is used for compiling TARGET MODULES. BUT ONLY if "USE_CLANG_QCOM := true" is set. Otherwise it is using aCLANG. So you can merge this commit and the build will be the same as before except you specifically define "USE_CLANG_QCOM := true"!

WARNING READ CAREFULLY: It is possbile that using this toolchain, especially with the optimizations flags, will introduce BUGS!
DONT SPAM QUALCOMM FORUM PLEASE!

I would suggest: IF YOU USE IT DONT MAKE BUG REPORTS FOR YOUR ROM OR KERNEL OR ANY APPS. Make reports in this development thread. The ROM compiles, boots and starts. Also if Bionic and other modules are compiled with Qualcomm Snapdragon LLVM Compiler 3.7.1 (qCLANG).
Androbench, Antutu, Basemark 2.0 , Geekbench 3, Vellamo are 'completing' the benchmarks. This is a work in progress. There is confirmation that it works for Sony Z Ultra and Samsung Galaxy Note 4 and after some months of testing everything seems good on my personal device.

At the moment this is only tested with Qualcomm Snapdragon 800 but in theory this should also work with any Krait and even Scorpion CPU. According to Documentation there is also support for armv8/aarch64 (Snapdragon 810) and you should gain a real performance bump but support is not yet implemented! If you have a Snapdragon 810 device and want to try this out contact me please.


Install instructions:
  1. You have to make an account in order to download the toolchain!
  2. You need to download the toolchain and extract it to {your android dir}/prebuilts/clang/linux-x86/host/*
  3. Merge/Cherry-Pick this commit: https://github.com/mustermaxmueller/compiler-rt/commit/774cabb50dced5b7acf62af331480397d3a0d12f
  4. Merge/Cherry-Pick this commit: https://github.com/mustermaxmueller/android_build/commit/894f0793d998395e0c15b9ad7a39b277e3aca006
  5. Merge/Cherry-Pick these commits:
    https://github.com/mustermaxmueller...mmit/05e2a338b2b66f30d86687c3de4e48d30a2db8d4
    https://github.com/mustermaxmueller...mmit/5f4eadfe1bbb5184d650cc9eef8f2306fefc7cdf
    You need the Qualcomm Snapdragon LLVM Compiler 3.5 for this to work!!!
    OR
    Make changes to frameworks/rs/driver/runtime/build_bc_lib_internal.mk like stated here: http://xdaforums.com/showpost.php?p=59626153&postcount=47
  6. Merge/Cherry-Pick this commit: https://github.com/jgcaaprom/android_prebuilt_ndk/commit/98100b2cbeb681698be6093ef964bd1f4414a71f
  7. Set "USE_CLANG_QCOM := true" e.g. in Boardconfig.mk
    OPTIONAL (set these options in your Boardconfig.mk):
    USE_CLANG_QCOM_VERBOSE := true (If you want to be certain that qclang is used. You will see extra verbose output while compiling.)
    USE_CLANG_QCOM_LTO := true (use LTO; not working at the moment)
    CLANG_QCOM_COMPILE_ART := true (compile art with qclang)
    CLANG_QCOM_COMPILE_BIONIC := true (compile bionic with qclang)
    CLANG_QCOM_COMPILE_MIXED := true (compile defined modules with qclang)
    CLANG_QCOM_USER_MODULES += (Here you can overwrite which modules should be compiled with qCLANG instead of GCC)
    CLANG_QCOM_FORCE_COMPILE_ACLANG_MODULES += (Here you can overwrite which modules should be compiled with aCLANG instead of qCLANG)
  8. Make a clean compile! Also delete ccache!


Annotation:
  1. First boot and every boot after cleaning cache will take up to 15min! Just be patient even if it seems that device is stuck at optimizing apps.
  2. Should also work with Android 6.0 but not tested yet!
  3. You should only get measurable performance benefits if you compile Bionic with CLANG!
  4. Documentation for the toolchain: {your android dir}/prebuilts/clang/linux-x86/host/llvm-Snapdragon_LLVM_for_Android_3.7/prebuilt/linux-x86_64Snapdragon_LLVM_ARM_37_Compiler_User_Guide.pdf
  5. The Flags "-muse-optlibc" and "-ffuse-loops" are not Documented. (http://pastebin.com/PZtk6WHB)
  6. The implementation is not beautiful but it works
  7. HOST is compiled with aCLANG because qCLANG does not understand x86
  8. You can make performance and size comparisons by removing/commenting "USE_CLANG_QCOM := true"
  9. I could have made the installation easier by uploading the toolchain to Github but I do not know if I am allowed to. And I am no lawyer so...

TODO:
-Make more performance comparisons to aCLANG and GCC.
-Compile Android with qCLANG wherever possible. http://www.linuxplumbersconf.org/20...ations/1131/original/LP2013-Android-clang.pdf and https://events.linuxfoundation.org/sites/events/files/slides/2014-ABS-LLVMLinux.pdf


Development Thread: http://xdaforums.com/android/software-hacking/wip-compile-android-5-0-2-qualcomm-llvm-t3035162


Credits:
-@Cl3Kener for looking at the first commit
-@the people whom stuff i used
-@XDA where i learned alot
-@sombree for finding out that you need libllvm3.6 (obsolete)

XDA:DevDB Information
Compile Android with Qualcomm LLVM 3.7.1 for devices with Qualcomm CPU, Tool/Utility for all devices (see above for details)

Contributors
MusterMaxMueller

Version Information
Status: Testing

Created 2015-02-19
Last Updated 2015-11-22
 
Last edited:

Alx31

Senior Member
May 15, 2013
5,583
8,370
43
Let me be the first to say CONGRATULATIONS FOR THIS, u rock man =) lets try these things, and thank you for sharing!
 
  • Like
Reactions: Odin Erbe

frap129

Inactive Recognized Developer
Jan 6, 2013
1,392
4,126
I've disabled clang for art and bionic entirely, so it just compiles with GCC. I wonder which would be better, since compiling with GCC allows for more optimizations such as graphite.
 

MusterMaxMueller

Senior Member
Mar 12, 2014
224
191
I've disabled clang for art and bionic entirely, so it just compiles with GCC. I wonder which would be better, since compiling with GCC allows for more optimizations such as graphite.

Yes, we have to measure and compare :)
I hope together we can get the manpower to verify/test what is better. That is why i started this thread.
According to linaro, graphite doesnt make that big difference. too lazy to search for the link right now.

I suck at c and the compiler stuff but in my understanding the LTO implementation of CLANG also sounds very interesting.
will update OP.
 
  • Like
Reactions: frap129

frap129

Inactive Recognized Developer
Jan 6, 2013
1,392
4,126
Yes, we have to measure and compare :)
I hope together we can get the manpower to verify/test what is better. That is why i started this thread.
According to linaro, graphite doesnt make that big difference. too lazy to search for the link right now.

I suck at c and the compiler stuff but in my understanding the LTO implementation of CLANG also sounds very interesting.
will update OP.

Well, Graphite does somewhat improve memory(RAM) use, and using the latest ISL and CLooG (as seen in sabermod) has only increased this. There are other options, for example Polly (The clang/LLVM equivalent of graphite) could be used for better optimization. And I didnt know that Clang implemented LTO differently, Ill look into that.
 

frap129

Inactive Recognized Developer
Jan 6, 2013
1,392
4,126
fixed some stuff and may have a solution for compiler-rt and libc++
if everything works well i will update in 3h.
I haven't gotten around to trying the SnapDragon LLVM, but since Qualcomm didn't release their source code, I compiled my own LLVM using stock LLVM sources with AOSP patches and some extra features enabled such a polly and libffi. https://github.com/frap129/host-llvm_3.5-prebuilt.git There's the link if anyone wants to test.
 

frap129

Inactive Recognized Developer
Jan 6, 2013
1,392
4,126
Got around to implementing this into my builds, and it seems that Qcom LLVM has all the same problems as my LLVM that I compiled (host not working, compiler_rt problems).
EDIT: Also, testing Link Time Optimizations (LTO)
 
Last edited:
  • Like
Reactions: MusterMaxMueller

MusterMaxMueller

Senior Member
Mar 12, 2014
224
191
Got around to implementing this into my builds, and it seems that Qcom LLVM has all the same problems as my LLVM that I compiled (host not working, compiler_rt problems).
EDIT: Also, testing Link Time Optimizations (LTO)

All fixed :) well except for HOST which wont work with qCLANG. look at my new commit. im sure will help you with your problems.
There were also some derps...
will update OP. already uploaded to github. took longer then i thought. my pc needs more ram.
also did some performance and stability testing. all seems good so far :)

will upload proof-of-concept rom with good performance. but building takes some time. need more ram :)

if you have llvmgold.so compiled would you mind sharing?
 
  • Like
Reactions: frap129 and Alx31

frap129

Inactive Recognized Developer
Jan 6, 2013
1,392
4,126
All fixed :) well except for HOST which wont work with qCLANG. look at my new commit. im sure will help you with your problems.
There were also some derps...
will update OP. already uploaded to github. took longer then i thought. my pc needs more ram.
also did some performance and stability testing. all seems good so far :)

will upload proof-of-concept rom with good performance. but building takes some time. need more ram :)

if you have llvmgold.so compiled would you mind sharing?

Nope, but I can compile it if you want. Also, for the LTO implementation, add the flag -lto as well, and make sure you add it to linker (LD ) flags
 

WhiteNeo

Senior Member
Feb 13, 2013
2,448
4,097
I have just compiled CM12 with QCOM llvm.. Build is booting and running fine, however the android progress circle indicator has some occasional animation issues that were not present before.

Might check the commit later.. Maybe an Ofast flag is the reason for this?
 
  • Like
Reactions: MusterMaxMueller

MusterMaxMueller

Senior Member
Mar 12, 2014
224
191
I have just compiled CM12 with QCOM llvm.. Build is booting and running fine, however the android progress circle indicator has some occasional animation issues that were not present before.

Might check the commit later.. Maybe an Ofast flag is the reason for this?

Thanks for trying and reporting back.
I have the same issues with animation circle. I think it might be fvectorize-loops but could be Ofast and i have no idea which module is causing this. Maybe libpng.
in the past i had the same problem with one of archidroids optimizations but cant remember which one it was :)

still having performance issues with art if compiled with qCLANG. made some changes and will test tonight or tomorrow.
 

WhiteNeo

Senior Member
Feb 13, 2013
2,448
4,097
Thanks for trying and reporting back.
I have the same issues with animation circle. I think it might be fvectorize-loops but could be Ofast and i have no idea which module is causing this. Maybe libpng.
in the past i had the same problem with one of archidroids optimizations but cant remember which one it was :)

still having performance issues with art if compiled with qCLANG. made some changes and will test tonight or tomorrow.
Got the animation issue fixed by removing all math related flags like -ffast-math and -funsafe-math-optimizations..
Also, I've replaced all -Ofast flags with -O3 (because Ofast includes the ffast-math flag, which is the issue here)
Will check later which flags may be reapplied without causing the issue again.

The build has been running smooth and stable for more than 24 hours. However, I can't report an extremely big difference in ui responsiveness. But the rom zip size has increased by 2mb, so the changes have indeed been applied. :good:
 
  • Like
Reactions: MusterMaxMueller

MusterMaxMueller

Senior Member
Mar 12, 2014
224
191
Got the animation issue fixed by removing all math related flags like -ffast-math and -funsafe-math-optimizations..
Also, I've replaced all -Ofast flags with -O3 (because Ofast includes the ffast-math flag, which is the issue here)
Will check later which flags may be reapplied without causing the issue again.

The build has been running smooth and stable for more than 24 hours. However, I can't report an extremely big difference in ui responsiveness. But the rom zip size has increased by 2mb, so the changes have indeed been applied. :good:

thanks for testing!
could you maybe test which module is causing the problems? so we can only disable fast_math where its needed. i guess it might be libpng?

regarding perfomance:
did you use my second commit? this should have some minor improvements in some cases. but actually decrease in some others... i need to look up how to make better and automated benchmarking. the way i do it at the moment is to time consuming.
anyone knows something i could use for automated benchmarking? for example just a zip i could copy/flash and then run via terminal? dont want to have to write my own :(
or something else, any suggestions?
i made some changes and am now compiling with the qcom libs which comes with the toolchain and finally got the special optimized copy function working. see "-arm-opt-memcpy" in the doc.
i hope this will give some performance improvements. wish some luck
at first it gave some problem because everytime i use the qcom libs i have to make sure libgcc isnt used.
i try to think about a cleaner solution because at the moment the way i do it is a dirty hack :)
then i will upload to github
 

WhiteNeo

Senior Member
Feb 13, 2013
2,448
4,097
thanks for testing!
could you maybe test which module is causing the problems? so we can only disable fast_math where its needed. i guess it might be libpng?

regarding perfomance:
did you use my second commit? this should have some minor improvements in some cases. but actually decrease in some others... i need to look up how to make better and automated benchmarking. the way i do it at the moment is to time consuming.
anyone knows something i could use for automated benchmarking? for example just a zip i could copy/flash and then run via terminal? dont want to have to write my own :(
or something else, any suggestions?
i made some changes and am now compiling with the qcom libs which comes with the toolchain and finally got the special optimized copy function working. see "http://xdaforums.com/showthread.php?t=3038250" in the doc.
i hope this will give some performance improvements. wish some luck
at first it gave some problem because everytime i use the qcom libs i have to make sure libgcc isnt used.
i try to think about a cleaner solution because at the moment the way i do it is a dirty hack :)
then i will upload to github

I used the two commits in OP. :)
libpng can't be the reason for this, as many Lollipop drawables, including the animation were talking about, are actually vector drawables, not png ones.

So the module that these vector drawables refer to returns broken numbers that make the animation look weird.

I think the safest way would be to skip ffast-math and Ofast completely, because the animation might just be THE ONE bug we actually see and notice. I don't want to imagine what else might happen in certain background processes, including the risk of broken data and random bugs.

By the way, looking forward to testing the qcom libs, along with lto. :)
How to add lto to the build?
 

Top Liked Posts

  • There are no posts matching your filters.
  • 27
    Use Qualcomm Snapdragon LLVM Compiler 3.7.1 (https://developer.qualcomm.com/mobi...-performance/snapdragon-llvm-compiler-android) for Android Lollipop 5.x for devices with Qualcomm CPU

    As far as I know nobody has done that yet. Except me :)

    This changes the toolchain for CLANG from the default one which comes with Android (aCLANG) to the Qualcomm Snapdragon LLVM Compiler 3.7.1 (qCLANG) mentioned above. The toolchain will be used for almost all places where the default Android CLANG (aCLANG) is used for compiling TARGET MODULES. BUT ONLY if "USE_CLANG_QCOM := true" is set. Otherwise it is using aCLANG. So you can merge this commit and the build will be the same as before except you specifically define "USE_CLANG_QCOM := true"!

    WARNING READ CAREFULLY: It is possbile that using this toolchain, especially with the optimizations flags, will introduce BUGS!
    DONT SPAM QUALCOMM FORUM PLEASE!

    I would suggest: IF YOU USE IT DONT MAKE BUG REPORTS FOR YOUR ROM OR KERNEL OR ANY APPS. Make reports in this development thread. The ROM compiles, boots and starts. Also if Bionic and other modules are compiled with Qualcomm Snapdragon LLVM Compiler 3.7.1 (qCLANG).
    Androbench, Antutu, Basemark 2.0 , Geekbench 3, Vellamo are 'completing' the benchmarks. This is a work in progress. There is confirmation that it works for Sony Z Ultra and Samsung Galaxy Note 4 and after some months of testing everything seems good on my personal device.

    At the moment this is only tested with Qualcomm Snapdragon 800 but in theory this should also work with any Krait and even Scorpion CPU. According to Documentation there is also support for armv8/aarch64 (Snapdragon 810) and you should gain a real performance bump but support is not yet implemented! If you have a Snapdragon 810 device and want to try this out contact me please.


    Install instructions:
    1. You have to make an account in order to download the toolchain!
    2. You need to download the toolchain and extract it to {your android dir}/prebuilts/clang/linux-x86/host/*
    3. Merge/Cherry-Pick this commit: https://github.com/mustermaxmueller/compiler-rt/commit/774cabb50dced5b7acf62af331480397d3a0d12f
    4. Merge/Cherry-Pick this commit: https://github.com/mustermaxmueller/android_build/commit/894f0793d998395e0c15b9ad7a39b277e3aca006
    5. Merge/Cherry-Pick these commits:
      https://github.com/mustermaxmueller...mmit/05e2a338b2b66f30d86687c3de4e48d30a2db8d4
      https://github.com/mustermaxmueller...mmit/5f4eadfe1bbb5184d650cc9eef8f2306fefc7cdf
      You need the Qualcomm Snapdragon LLVM Compiler 3.5 for this to work!!!
      OR
      Make changes to frameworks/rs/driver/runtime/build_bc_lib_internal.mk like stated here: http://xdaforums.com/showpost.php?p=59626153&postcount=47
    6. Merge/Cherry-Pick this commit: https://github.com/jgcaaprom/android_prebuilt_ndk/commit/98100b2cbeb681698be6093ef964bd1f4414a71f
    7. Set "USE_CLANG_QCOM := true" e.g. in Boardconfig.mk
      OPTIONAL (set these options in your Boardconfig.mk):
      USE_CLANG_QCOM_VERBOSE := true (If you want to be certain that qclang is used. You will see extra verbose output while compiling.)
      USE_CLANG_QCOM_LTO := true (use LTO; not working at the moment)
      CLANG_QCOM_COMPILE_ART := true (compile art with qclang)
      CLANG_QCOM_COMPILE_BIONIC := true (compile bionic with qclang)
      CLANG_QCOM_COMPILE_MIXED := true (compile defined modules with qclang)
      CLANG_QCOM_USER_MODULES += (Here you can overwrite which modules should be compiled with qCLANG instead of GCC)
      CLANG_QCOM_FORCE_COMPILE_ACLANG_MODULES += (Here you can overwrite which modules should be compiled with aCLANG instead of qCLANG)
    8. Make a clean compile! Also delete ccache!


    Annotation:
    1. First boot and every boot after cleaning cache will take up to 15min! Just be patient even if it seems that device is stuck at optimizing apps.
    2. Should also work with Android 6.0 but not tested yet!
    3. You should only get measurable performance benefits if you compile Bionic with CLANG!
    4. Documentation for the toolchain: {your android dir}/prebuilts/clang/linux-x86/host/llvm-Snapdragon_LLVM_for_Android_3.7/prebuilt/linux-x86_64Snapdragon_LLVM_ARM_37_Compiler_User_Guide.pdf
    5. The Flags "-muse-optlibc" and "-ffuse-loops" are not Documented. (http://pastebin.com/PZtk6WHB)
    6. The implementation is not beautiful but it works
    7. HOST is compiled with aCLANG because qCLANG does not understand x86
    8. You can make performance and size comparisons by removing/commenting "USE_CLANG_QCOM := true"
    9. I could have made the installation easier by uploading the toolchain to Github but I do not know if I am allowed to. And I am no lawyer so...

    TODO:
    -Make more performance comparisons to aCLANG and GCC.
    -Compile Android with qCLANG wherever possible. http://www.linuxplumbersconf.org/20...ations/1131/original/LP2013-Android-clang.pdf and https://events.linuxfoundation.org/sites/events/files/slides/2014-ABS-LLVMLinux.pdf


    Development Thread: http://xdaforums.com/android/software-hacking/wip-compile-android-5-0-2-qualcomm-llvm-t3035162


    Credits:
    -@Cl3Kener for looking at the first commit
    -@the people whom stuff i used
    -@XDA where i learned alot
    -@sombree for finding out that you need libllvm3.6 (obsolete)

    XDA:DevDB Information
    Compile Android with Qualcomm LLVM 3.7.1 for devices with Qualcomm CPU, Tool/Utility for all devices (see above for details)

    Contributors
    MusterMaxMueller

    Version Information
    Status: Testing

    Created 2015-02-19
    Last Updated 2015-11-22
    7
    Please see the following guide for how to compile Android Nougat using Snapdragon LLVM:
    https://developer.qualcomm.com/forum/qdn-forums/software/snapdragon-llvm-compiler-android/33437
    5
    Does this guide apply to Android 6.0? I know there are some changes that were done to compiler-rt from lollipop to marshmallow

    no not yet, got my device finally running on mm. i hope i can port the compile script to mm in the comming weeks.
    4
    Big Update

    Hi all,
    uploaded big update to github.

    Last update was messed up!!! i uploaded wrong branch but didnt realized it until today. it might work but i dont know.
    Big sorry for all that had problems because of that :(
    So revert the changes and get the new one.

    New update is working 100%. made build yesterday and everything is working fine!
    now even compiling frameworks rs wit qclang.
    Also good performance boost if you compile bionic with qclang.

    See OP for updated instructions!!!

    If you have any questions feel free to ask.