Set Up Your MediaTek Device with Comprehensive Beginner’s Guide

As can be seen with the use of MediaTek chips in Android One devices … more

Sony Updates AOSP Sources to Android 5.0.1

Just a few days ago, Sony did an utterly fantastic job by pushing out numerous device trees for … more

Stop Your Screen From Turning Off with KeepItOn

We all know the feeling of reading a news article orrather longdocument, when our screen … more

The XDA LG QPair Developer Challenge Voting Has Begun!

It seems like it was just yesterday that weannounced that we had paired up with LGto … more

Welcome to XDA

Search to go directly to your device's forum

Register an account

Unlock full posting privileges

Ask a question

No registration required
Post Reply

[DEV] How to compile from the source code for you Android

OP theGanymedes

5th January 2012, 07:27 PM   |  #1  
theGanymedes's Avatar
OP Senior Member
Flag Istanbul
Thanks Meter: 253
 
474 posts
Join Date:Joined: Oct 2010
Donate to Me
More
Greetings everyone...

As you might know: I'm the tech freak guy who compiled the GlibC and some Linux applications for HTC Desire.

As I have published my findings, I've recieved messages about how I'm compiling the applications given the source code. I try to help about this, but sadly I think half of what I say is not understandable without a proper guide. Ergo, this thread will suply this!

When?
Well, soon; because I'm still preparing it.

What will it include?
It'll be a basic guide about how to compile applications with your toolchain (Here is the guide about how to make a toolchain of yourself: How to make a toolchain). I'm planning to show it with GlibC compilation.

What's the extend of this help?
Frankly, compiling from the source sometimes can be quite messy and most of the time, a compilation/configuration line is never same with different applications. However, since most sources come with either Autotools or with a Makefile, most of the times, all you need is to change a few command line variable with a proper value.

I'm not planning to show you the code changes or "patchworks" they might be necessary, because this is quite application specific thing and I'm not an expert about this either. I'm just going to show you a guide about the compilation process.

What's the catch / scum of this help?
There is no catch in it. I just want to help curious nerdy guys like me (), so that we can learn from each other; or maybe improve each others work! I learned a lot from guys here, and I want to help to this community as much as I can as well.
The Following 8 Users Say Thank You to theGanymedes For This Useful Post: [ View ]
5th January 2012, 07:27 PM   |  #2  
theGanymedes's Avatar
OP Senior Member
Flag Istanbul
Thanks Meter: 253
 
474 posts
Join Date:Joined: Oct 2010
Donate to Me
More
Compilation Guide from Source code
Well, let's begin

We're going to compile the GlibC, with all internals to make it ready. After you prepare this, you might also check DoomLord's post here (http://forum.xda-developers.com/show....php?t=1041064) to make a recovery zip to flash this to your device. Make yourself comfortable now; because I'm going to explain everything we use here, thus this might be loong guide

1- Obtain the Sources

First, obtain the sources that you're going to compile. Check all the dependencies of the package and download them (and compile them) before. In our case, since we'll compile GlibC and all it needs is a working toolchain; we don't need to worry about this.

You can obtain GlibC source code from http://ftp.gnu.org/gnu/glibc/ address. Note that, since we're building GlibC for ARM devices and since GlibC seperated ARM support to "Ports" package, we need to download that as well. Make sure you download the same versions of Glibc and Glibc-Ports packages.



2- Extract the Sources, and make a build directory

Extract the sources to some folder; say /home/<user>/Desktop/glibc . Also extract the glibc-ports package to some direcorty, say /home/<user>/Desktop/glibc-ports.

After this, rename the folder "glibc-ports" to just "ports" and then move it inside to the "glibc" folder. If you don't do this step, you'll have "ARM is unsupported" error in configuration step.

Since glibc cannot be compiled from the directory which sources are in it (a restriction made by the developers, to make sources unchanged and ready to be compiled again), we need to make another folder from which we're going to call the compilation tools. Let's say this directory is named as "glibc-build", also at /home/<user>/Desktop .



3- Start "configure"

Now, open a terminal emulator window ( you can use Ctrl+Alt+T keys under Ubuntu to easily open one ). Change into the glibc-build directory.

Code:
 cd /home/<user>/Desktop/glibc-build


After this operation, we've to call the glibc's auto-configure script from this folder. Note that ".." is a "special folder name" which denotes the upper level directory (so basicly, we're changing into the upper level in directory tree - Desktop in our case - and then call something from a folder inside that)

Code:
 ../glibc-2.14/configure --help
When you run the above command, configure will give you the options you can use to configure this package for your needs. Nearly 90% of the time, checking the output of this command gives you the options you need to set to compile properly.

Code:
`configure' configures GNU C Library (see version.h) to adapt to many kinds of systems.

Usage: ../glibc-2.14/configure [OPTION]... [VAR=VALUE]...

To assign environment variables (e.g., CC, CFLAGS...), specify them as
VAR=VALUE.  See below for descriptions of some of the useful variables.

Defaults for the options are specified in brackets.

Configuration:
  -h, --help              display this help and exit
      --help=short        display options specific to this package
      --help=recursive    display the short help of all the included packages
  -V, --version           display version information and exit
  -q, --quiet, --silent   do not print `checking ...' messages
      --cache-file=FILE   cache test results in FILE [disabled]
  -C, --config-cache      alias for `--cache-file=config.cache'
  -n, --no-create         do not create output files
      --srcdir=DIR        find the sources in DIR [configure dir or `..']

Installation directories:
  --prefix=PREFIX         install architecture-independent files in PREFIX
                          [/usr/local]
  --exec-prefix=EPREFIX   install architecture-dependent files in EPREFIX
                          [PREFIX]

By default, `make install' will install all the files in
`/usr/local/bin', `/usr/local/lib' etc.  You can specify
an installation prefix other than `/usr/local' using `--prefix',
for instance `--prefix=$HOME'.

For better control, use the options below.

Fine tuning of the installation directories:
  --bindir=DIR            user executables [EPREFIX/bin]
  --sbindir=DIR           system admin executables [EPREFIX/sbin]
  --libexecdir=DIR        program executables [EPREFIX/libexec]
  --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
  --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
  --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
  --libdir=DIR            object code libraries [EPREFIX/lib]
  --includedir=DIR        C header files [PREFIX/include]
  --oldincludedir=DIR     C header files for non-gcc [/usr/include]
  --datarootdir=DIR       read-only arch.-independent data root [PREFIX/share]
  --datadir=DIR           read-only architecture-independent data [DATAROOTDIR]
  --infodir=DIR           info documentation [DATAROOTDIR/info]
  --localedir=DIR         locale-dependent data [DATAROOTDIR/locale]
  --mandir=DIR            man documentation [DATAROOTDIR/man]
  --docdir=DIR            documentation root [DATAROOTDIR/doc/c-library]
  --htmldir=DIR           html documentation [DOCDIR]
  --dvidir=DIR            dvi documentation [DOCDIR]
  --pdfdir=DIR            pdf documentation [DOCDIR]
  --psdir=DIR             ps documentation [DOCDIR]

System types:
  --build=BUILD     configure for building on BUILD [guessed]
  --host=HOST       cross-compile to build programs to run on HOST [BUILD]

Optional Features:
  --disable-option-checking  ignore unrecognized --enable/--with options
  --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)
  --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]
  --disable-sanity-checks really do not use threads (should not be used except
                          in special situations) [default=yes]
  --enable-check-abi      do "make check-abi" in "make check" (no/warn/yes)
                          [default=no]
  --enable-shared         build shared library [default=yes if GNU ld & ELF]
  --enable-profile        build profiled library [default=no]
  --enable-omitfp         build undebuggable optimized library [default=no]
  --enable-bounded        build with runtime bounds checking [default=no]
  --disable-versioning    do not include versioning information in the library
                          objects [default=yes if supported]
  --enable-oldest-abi=ABI configure the oldest ABI supported [e.g. 2.2]
                          [default=glibc default]
  --enable-stackguard-randomization
                          initialize __stack_chk_guard canary with a random
                          number at program start
  --enable-add-ons[=DIRS...]
                          configure and build add-ons in DIR1,DIR2,... search
                          for add-ons if no parameter given
  --disable-hidden-plt    do not hide internal function calls to avoid PLT
  --enable-bind-now       disable lazy relocations in DSOs
  --enable-static-nss     build static NSS modules [default=no]
  --disable-force-install don't force installation of files from this package,
                          even if they are older than the installed files
  --enable-kernel=VERSION compile for compatibility with kernel not older than
                          VERSION
  --enable-all-warnings   enable all useful warnings gcc can issue
  --enable-multi-arch     enable single DSO with optimizations for multiple
                          architectures
  --enable-experimental-malloc
                          enable experimental malloc features
  --enable-nss-crypt      enable libcrypt to use nss

Optional Packages:
  --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
  --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
  --with-gd=DIR           find libgd include dir and library with prefix DIR
  --with-gd-include=DIR   find libgd include files in DIR
  --with-gd-lib=DIR       find libgd library files in DIR
  --with-fp               if using floating-point hardware [default=yes]
  --with-binutils=PATH    specify location of binutils (as and ld)
  --with-elf              if using the ELF object format
  --with-selinux          if building with SELinux support
  --with-xcoff            if using the XCOFF object format
  --without-cvs           if CVS should not be used
  --with-headers=PATH     location of system headers to use (for example
                          /usr/src/linux/include) [default=compiler default]
  --with-tls              enable support for TLS
  --without-__thread      do not use TLS features even when supporting them
  --with-cpu=CPU          select code for CPU variant

Some influential environment variables:
  CC          C compiler command
  CFLAGS      C compiler flags
  LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
              nonstandard directory <lib dir>
  LIBS        libraries to pass to the linker, e.g. -l<library>
  CPPFLAGS    (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
              you have headers in a nonstandard directory <include dir>
  CPP         C preprocessor
  CXX         C++ compiler command
  CXXFLAGS    C++ compiler flags

Use these variables to override the choices made by `configure' or to help
it to find libraries and programs with nonstandard names/locations.

Report bugs to <glibc>.
GNU C Library home page: <http://www.gnu.org/software/c-library/>.
General help using GNU software: <http://www.gnu.org/gethelp/>.
The help is quite straightforward, and all explanations are already done. The most important options for our case here is "prefix" and "host". Also, for compilation of GlibC, we need to give "with-headers", "with-tls" and "with-ports" proper values.

"Prefix" is a very important option. This options specifies where the installed binaries and libraries "think" they are. Some binaries and libs. are quite flexible: they don't care about where they are, thus the value of prefix isn't much of importance, but most of the time, it is. Omitting a prefix value here will make PREFIX value of "/usr/local" - which is quite dangerous because it may screw your own system (/usr/local is usually where your own PC applications are - most probably you cannot screw this folder if you're not a super-user but, well, anyway...); so always change this value to something proper when you're cross compiling.

Also, make sure that this path exists in your target device. If you set a prefix, say /cache, and then copy the stuff at, say, /data; it's highly probable that your applications will search for it's config files at /cache, and will complain that they are not found - since they're at /data. Since AFAIK, there is no way to change this in most application after compilation of software, you have to recompile the program to fix such errors (or move the application to the prefix it's set).

I usually use /data as prefix, because I install the stuff to there.

Host is another very important option. If you don't give this parameter, auto-configure will think you're compiling this application for the PC you're using. We've to give "host" the machine specification we're compiling the app for. For ARM based devices, this is mostly something line "arm-xxx-linux" or "arm-linux" or "arm-xxxx-linux-gnueabi" or such.

How do you found this out? Well, simple: If you followed my crostool-ng tutorial and used the same options as I did; your host name will be in "arm-xxxxxx-linux-gnueabi" format. "xxxxx" here is what your vendor name is. If you set a vendor value when making the toolchain, you must write it here. If you left it empty, xxxxx will be "unknown".

The other way to check this is your toolchain's name of gcc command: If your ARM GCC toolchain name is "arm-myvendor-linux-gnueabi-gcc" then, your host name is "arm-myvendor-linux-gnueabi".

Most of the times, there are other options that you must give. As for an example, here is what I generally use to compile glibC:

Code:
../glibc-2.14/configure --host=arm-msm-linux-gnueabi --prefix=/data --with-tls --with-ports="nptl, ports"
I use host as "arm-msm-linux-gnueabi" because this is my cross compile target name.

We set "with-tls" option here: it's for GlibC to support Thread Local Storage. This is a programming scheme that allows threads to have their own storage area; and we give this configuration here to give GlibC a support for this. More info about TLS can be found online in programming wiki's.

We also set "with-ports" because we're also installing some extra plugins for our GlibC install. NPTL is an advanced threading library that supports many advanced features like TLS and such. There is also a "linuxthreads" but it's older. New versions of GlibC is not shipped with "linuxthreads" either. The other port, "ports" is for ARM support.

You must also give the kernel source directory here with a parameter of "with-headers" if you're using some other toolchain and your kernel sources are in some other place. Thanks to crostool-ng that it moves all the headers to the toolchain folder and makes them automatically reachable, so you can omit this parameter.

Well, when you give the command and supply the options and press ENTER, it shall start configuring the package:


When there is a missing dependency or option, it shall generally give you a warning at this step and terminate the configuration. You must then find the solution of this error (install the necessary dependency etc.) and then try to configure the package again - as I said, this is usually not the case for GlibC .

3- Start "make"

After configuration process is done without errors, you can pass to the compilation phase. If you did configuration properly, you don't have to use special parameters and such for most packages. Just issue the command "make" and wait

Code:
make
Once this is completed without errors, you're nearly ready for the shipment Note that, if you get any errors in this process, you should definitely google it; since errors in this level is quite application and build environment (your machines configs. etc.) specific. The errors in this phase is usually needs some tricks to fix, and they are usually not so easy to fix

3- Start "make install"

Once the compilation is successful, you can install the application to the "PREFIX" folder you specified whilst configuring the package. In order to do that, just issue:

Code:
make install
However, issuing just this command isn't what we want generally: Since we've cross compiled the application, we're not interested in installing that to our PC! We want it to be somewhere we can easily access, so that we can distribute easily right?

For most packages, three important variables are necessary to make this. Which variable is heeded is highly dependent of package, however it's DESTDIR usually used.

DESTDIR: The variable usually used in makefiles. When you set DESTDIR while giving "make install", the application becomes installed to DESTDIR/PREFIX folder, instead of just PREFIX folder without changing the prefix info inside the application so it makes it easier for us to distribute the package.

install_root : GlibC uses this variable, instead of DESTDIR. I didn't see any other packages using this.

INSTALL_TOP: OpenSSL and Lua packages use this variable instead of DESTDIR.

Which of these variable is used cannot be identified either without examining the Makefile, or without testing it

How are you going to give it a value? Simple: you just assign the variable a value, after giving "make install" command:

Code:
make install DESTDIR=/home/<user>/myapp
For GlibC, this becomes:

Code:
make install install_root=/home/<user>/glibc
Which makes glibc installed to /home/<user>/glibc folder.

You can take the files from this folder and make a flashable zip; and send it to your device!

5- NOTES

- Be careful about the PREFIX value, you should install the files to this place in your device as well.

- Most of the times, you don't need the "include" folder - this folder keeps the header files for your packages, so that other applications may be compiled with them. Since we're not compiling applications in our device, we don't need headers. Compiled applications don't use headers anymore.

- Most of the times, .a files under the "lib" folder is unnecessary - these are static libraries that are usually used when applications are being compiled. Since we don't compile stuff at our device, we don't need them - they are usually quite big too! Be careful though: some packages don't offer .so files (which are dynamic libraries for applications to use dynamically) - it might be necessary to keep .a files then.

- .la files are needed for application compilation with libraries, you can erase them as well.

- pkgconfig can be erased in distribution packages. PKGConfig is a tool for compilation that automatically parses the files in pkgconfig dirs to give necessary compilation parameters with dynamic libraries to the applications at compile time. Like we don't need headers, we don't need those files.

Well, that's it. I hope I could be of some help. See you next time and happy Android'ing!
Attached Thumbnails
Click image for larger version

Name:	tutorial1.jpg
Views:	895
Size:	11.2 KB
ID:	843800   Click image for larger version

Name:	tutorial2.png
Views:	2471
Size:	55.5 KB
ID:	843801   Click image for larger version

Name:	tutorial3.png
Views:	2458
Size:	13.4 KB
ID:	843802   Click image for larger version

Name:	tutorial4.jpg
Views:	2462
Size:	46.5 KB
ID:	843803  
Last edited by theGanymedes; 5th January 2012 at 09:57 PM. Reason: Guide is written
The Following 5 Users Say Thank You to theGanymedes For This Useful Post: [ View ]
5th January 2012, 07:28 PM   |  #3  
theGanymedes's Avatar
OP Senior Member
Flag Istanbul
Thanks Meter: 253
 
474 posts
Join Date:Joined: Oct 2010
Donate to Me
More
< Reserved for other stuff for guide, like hints.. >
The Following 2 Users Say Thank You to theGanymedes For This Useful Post: [ View ]
6th January 2012, 05:24 AM   |  #4  
Droidzone's Avatar
Recognized Developer
Flag Kerala
Thanks Meter: 2,217
 
5,471 posts
Join Date:Joined: Sep 2010
Donate to Me
More
Maybe also include a precompiled toolchain for download?
6th January 2012, 01:20 PM   |  #5  
theGanymedes's Avatar
OP Senior Member
Flag Istanbul
Thanks Meter: 253
 
474 posts
Join Date:Joined: Oct 2010
Donate to Me
More
Can do that, but first need to compile a "static" toolchain for this. Actually, this seems like a good idea

Let me work on this next
6th January 2012, 02:23 PM   |  #6  
Alex-V's Avatar
Recognized Developer
Thanks Meter: 4,558
 
8,335 posts
Join Date:Joined: Aug 2008
More
Nice...big thx...i love such guide´s

with kind regards...Alex
6th January 2012, 06:27 PM   |  #7  
theGanymedes's Avatar
OP Senior Member
Flag Istanbul
Thanks Meter: 253
 
474 posts
Join Date:Joined: Oct 2010
Donate to Me
More
Quote:
Originally Posted by theGanymedes

Can do that, but first need to compile a "static" toolchain for this. Actually, this seems like a good idea

Let me work on this next

Tried that: The output is huge, and even though the toolchain works without any need to any library, it still needs the libraries to be compiled in order to compile the other binaries - makes sense, since you need the libraries to compile against them anyway
6th January 2012, 07:05 PM   |  #8  
Droidzone's Avatar
Recognized Developer
Flag Kerala
Thanks Meter: 2,217
 
5,471 posts
Join Date:Joined: Sep 2010
Donate to Me
More
So isn't it possible to package your toolchain which contains gcc, initial glibc and libraries, so that we can download and use them right away? It would save a lot of time and would be a far better option than the ndk.

Sent from my HTC Desire using Tapatalk
6th January 2012, 08:13 PM   |  #9  
theGanymedes's Avatar
OP Senior Member
Flag Istanbul
Thanks Meter: 253
 
474 posts
Join Date:Joined: Oct 2010
Donate to Me
More
Quote:
Originally Posted by Droidzone

So isn't it possible to package your toolchain which contains gcc, initial glibc and libraries, so that we can download and use them right away? It would save a lot of time and would be a far better option than the ndk.

Sent from my HTC Desire using Tapatalk

That's possible, but only if you're also using Ubuntu 10.10 or 10.04 and a 64-bit machine

For some reason - guess library version differences - toolchains done in 11.10 was not working at 10.04; so I had to redo it, for instance. This technique you say might or might not work - there is no guarantee in it.
6th January 2012, 08:17 PM   |  #10  
Droidzone's Avatar
Recognized Developer
Flag Kerala
Thanks Meter: 2,217
 
5,471 posts
Join Date:Joined: Sep 2010
Donate to Me
More
Ah well.. I'm using a 32 bit Ubuntu 11.10 on a 64 bit machine. Anyway I have the toolchain I compiled on this one with your help..But if you remember I took a couple of days to get it done, even when I had your excellent help..Newcomers might find it difficult I guess. But if it werent for your toolchain compilation guide in the glibc thread, it'd have been next to impossible!
Last edited by Droidzone; 6th January 2012 at 08:21 PM.

Post Reply Subscribe to Thread

Tags
arm, compilation, toolchain
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes