5,597,762 Members 36,538 Now Online
XDA Developers Android and Mobile Development Forum

[SCRIPTSET v2.6][02/28/2012] Build native GNU/Linux applications: the easy way

Tip us?
 
smitna
Old
(Last edited by smitna; 29th February 2012 at 05:23 PM.)
#1  
Senior Member - OP
Thanks Meter 110
Posts: 161
Join Date: Aug 2011

 
DONATE TO ME
Default [SCRIPTSET v2.6][02/28/2012] Build native GNU/Linux applications: the easy way

New: SANE (USB Scanner backend and drivers) - OTG equipped device needed to work for this - YES, Scanner Access Now possible and Easy with your phone

This can (in combination with tesseract-ocr) be very useful to recognize text from scanned images.

For pictures see also http://forum.xda-developers.com/showpost.php?p=23046120&postcount=187

Update 01/30/2012: Now with a small android app (ScriptStarter) to start services and other shell scripts easily from the GUI. See bottom of the post.


WHAT'S POSSIBLE WITH THIS SCRIPT:

With this bash script your are able to build relatively easy from source a lot of native software (over 90 at the moment) from the GNU/Linux community for your Android phone and you will have
the good experience with free software.
All these tools and programs are not "apps for the GUI" for Android but native running command line tools "under the surface" of your Android GUI like the kernel and other services.
Don't ask me if it makes any sense to install program xy (like MySQL) on a phone. This is your decision. It's possible, so try it and see what you can do with it.

Note: I tested most of the compiled software on a Samsung Galaxy S2 (my phone). I cannot test it on a variety of phones because I have no mobile phone shop...


Currently buildable with this scripts (abstract, see the full list in package_selection.conf):
  • PHP 5.3.10 (with almost all extensions and XCache)
  • MySQL 5.5.19 (if you want a full database server on your phone...)
  • lighttpd 1.4.30
  • OpenSSH 5.9p1
  • OpenSSL 1.0.0f
  • Samba 3.6.3
  • cURL 7.23.1
  • bash 4.2
  • BusyBox 1.19.3
  • ImageMagick 6.7.4-10
  • OpenVPN 2.2.2
  • Python 2.7.2
  • Node.js 0.6.7
  • git 1.7.9
  • Wget 1.13.4
  • Rsync 3.0.9
  • TCPDUMP 4.2.0
  • Midnight Commander 4.8.1
  • Pure-FTPd 1.0.35
  • screen 4.0.3
  • iptables 1.4.10 (without bionic bug "getnetbyaddr()" and with iptables-save/restore!)
  • BIND 9.8.1-P1
  • Lua 5.2.0
  • Ruby 1.9.3-p0
  • usbutils 004
  • MTDUtils latest git
  • GCC 4.6.2 (c, c++)
  • GNU Coreutils 8.14, Binutils 2.22, Findutils 4.4.2, Inetutils 1.8
  • QEMU 0.15.1
  • SANE 1.0.22
  • tesseract-ocr 3.01
  • ...
  • a lot of libraries, development tools and other useful programs
    ...


FEATURES:
  • automatic download and untar the cross-compilers (if not already done) for cross compiling
  • automatic download and untar the source packages (if not already done), that you have selected
  • solve dependences and reserves additional needed packages
  • apply patches to the sources if necessary
  • build them, "install" them in a folder for later "ready to transfer" to your phone and stripes the binaries and libraries to reduce the filesize if selected
  • autogenerate a tar file and adb push the file to device if selected
  • allows you to use well-known software like PHP, MySQL, OpenSSL, OpenSSH, OpenVPN, bash and lighttpd on your phone (see the full list in file package_selection.conf, see attachement)
  • scripting and development on your phone
  • no limits with BusyBox (without want to be derogative, BusyBox is a wonderful tool), full featured tools like Coreutils, Binutils etc.
  • build scripts are expandable with other or newer packages you eventual want to have

QUESTIONS AND ANSWERS:

Q: Do I need native GNU/Linux programs and what is that?
A: If you do not know what native Linux applications are, then stop here - you don't need them.

Q: What can I do with these tools?
A: Many things. As a developer you can use the libraries to include them in Android GUI apps, as user you can start for example a webserver on your phone and write some HTML or PHP pages
just for fun or replace BusyBox with full featured tools (or only some commands).
Some tools are also very useful to find network errors, check the network traffic and many more.

Q: What do I need?
A: A Linux installation (VM or real), some Linux knowledge and a little bit time to read all instructions carefully. A rooted Android device with enouth free memory (ext2/3/4 formatted)
See README for details.

Q: Costs?
A: Nothing. The scripts are Open Source (GPL v3), the packages are GPL licensed or under a similar license. The cross-compilers are Open Source or free to use for non-profit purposes.
But I would be pleased if I get a donation and/or a press on the "Thanks button", if the scripts are useful for you. It was a long and hard work...


Note: The attachement scriptset-x.yy.zip is double packed (inside .bz2), because I want preserve the right settings of the scripts.

STEPS:
  1. Selected the desired packages in the config file
  2. Adjust the compiler settings for your phone
  3. Run the build script to download and compile the software
  4. Copy the generated TAR files to your phone or let "adb" this do for you and unpack them in the prepared sysroot folder

For more information please read the instructions in "README"
Please let me also know if you have any problems or if you have questions or further suggestions.

Thanks go to the Open Source community and a lot of people/webpages with hints about cross-compiling software.

Thanks

smitna

CHANGELOG:
01/14/2012 First public release v0.9
01/14/2012 Update: Removed unneeded big patch for ncurses
01/15/2012 Release of v1.0
Update: Fixed build of gcc. Note: the header files are not copied automatically with the script.
If you want to compile on your phone, set TAR_HEADERS=1 in configuraton.conf
Typos: some translations corrected in build.sh
01/16/2012 Release of v1.1
Update: Removed building of fsck from util-linux-ng, it's in e2fsprogs
Update: ISC Cron / vixie cron: set correct paths in pathnames.h before compiling
Enhancement: More detailed explanation of device specific compiler settings with list of possible values
Update: Removed dependence e2fsprogs libs from e2fsprogs
01/17/2012 Release of v1.2
Update: Build of zip, zipcloak, zipnote, zipsplit, unzip, unzipsfx and funzip now possible without errors
01/17/2012 Release of v1.3
Update: Added Node.js
Typos: some translations and language errors corrected in build.sh and README (sorry, I'm not a native English speaker...)
01/18/2012 Release of v1.4
Fixed TMP_PATH in PHP extension XCache (mmap.c) to prevent startup warning
Fixed BusyBox problems with static build against glibc ("__getpagesize" segfault with ping, traceroute and other networking parts)
and added some BusyBox patches.
BusyBox is now dynamic linked, so don't replace the system BusyBox with this build, because of missing mount/paths to libraries at phone startup
01/20/2012 Release of v1.5
Update: Cleanup in scripts and separation of configuration and code. Now it's only necessary to edit configuration.conf and package_selection.conf
Update: Added gzip
Update: Added experimental adb push to device if the device is connected and permissions are ok
Update: Check for installed programs. Script now exists on check failure to minimize errors while compiling.
01/21/2012 Release of v1.6
Bugfix: MySQL build could fail if gen_lex_hash could not generate the file sql/lex_hash.h in pass 2
Update: adb push should now work, check for connected device (and possible multiple devices) and check for right permissions to use the device
Enhancement: Even more explanation of device specific compiler flags and some hints to find the right settings.
01/21/2012 Release of v1.7
Enhancement: To eliminate sure a failing MySQL build, with every new run the source folder will be removed before completely
Enhancement: file version is now strictly checked (the host version must be the same as the cross version)
Some minor changes and filechecks (e.g. autoconf)
01/23/2012 Release of v1.8
Bugfix: Python has SQLite dependencies and SQLite must be ready before Python configure/make
Bugfix: In certain circumstances the optimizer settings get lost
Bugfix: Removed --enable-multibyte from bash, no chance to use it with NDK
01/24/2012 Release of v1.9
Update: Added ngIRCd (Please read the comment for this server, because of running as root)
Update: Sorted the links and package names in the config a little bit...
01/26/2012 Release of v2.0
Bugfix: OpenSSH wants to touch /var/empty while compiling, should be fixed --with-privsep-path now set (thanks to member dateno1 for his reports of such warnings/errors)
Bugfix: Removed a wrong CFLAG -I path from NDK config (only useful for me, really useless for others...)
Enhancement: Example for a user db for Pure-FTPd virtual user database in README (please use virtual users, it's the best and easiest solution)
Update: OpenSSH build need ssh-keygen for the generation of the default keys, added this info to README and the package selection
Bugfix: Not really a bugfix, but some installers don't set the right permissions of libraries in the sysroot folder, so set them global in the final task
Bugfix: Removed --with-lastlog=$TARGET_SYSROOT/var/log/lastlog from OpenSSH and added --with-lastlog=no, I don't know why this is not working and throws a warning, but we can live without it...
Enhancement: Added option to choose the sysroot variant from the CodeSourcery libs: v5TE (default), v4T (old), thumb2 (v7, fastest if supported by the phone)
Enhancement: Added option to tar the headers for development on the device with CodeSourcery (hint: always set rpath in LDFLAGS to make use of the CodeSourcery libs)
01/26/2012 Release of v2.1
Update: Added getcap/setcap and libraries from wishlist, needs libtoolize. Note: This is in a testing stage, kernel and filesystem support needed!
01/30/2012 Added a ScriptStarter app
09/02/2012 Release of v2.2
Update: Local compilation of static "tic" from ncurses for buildhost, if ncurses installation is missing
Enhancement: "downloading only" mode without compiling the packages (and force download)
Enhancement: some parameters for starting the script with additional functions (show with ./build.sh help)
Enhancement: more logging and outputs
Enhancement: language specs, all outputs can be changed to other languages in build.sh
Update: Added dosfstools
Update: Added Ruby
Update: Added example configs and starter scripts for Samba and OpenSSH
Update: Use of http-URLs if possible and update of some broken external links since last release
Bugfix: util-linux-ng needs gettext, added dependence
Bugfix: Added parameter to Samba to solve user/group settings and socket_wrapper support to set path vars at runtime
Update: Changed PHP version to 5.3.10, Samba to 3.6.3 and git to 1.7.9
02/11/2012 internal Release v2.3
Bugfix: openVPN patch for /dev/tun and links to ip/netstat/ifconfig/route, see hints for symlinks
02/16/2012 internal Release v2.4
02/25/2012 Release of v2.5
Update: Added LAME
Update: Added experimental CUPS
Update: Changed PNG version to 1.5.9
Update: Added Net-SNMP
Update: Added Netpbm
Update: Added PNG 1.2.47 for Netpbm and phpSANE
Update: Added LibTIFF
Update: Added SANE (USB Scanner backend and drivers) - OTG equipped device needed to work for this - YES, Scanner Access Now possible and Easy with your phone
Update: Split patches for bash (4.1 and 4.2) for better and easier (automatic) inclusion of original patches
02/28/2012 Release of v2.6
Update: Added Cuneiform OCR
Update: Added tesseract-ocr (see hints for more details howto use it)

ScriptStarter
02/03/2012 Added a small ScriptStarter for the GUI. With this app, written with JQuery, PhoneGap and Java (wrapper to execute the shell scripts) it's possible to start for example the lighttpd web server, Pure-FTPd or MySQL from within the Android GUI without a terminal and read the output. If a script needs root access, there is a checkbox to allow this. For this to work, copy the script sush in the script folder. All scripts ending with .sh can be started (please choose after the first start your script folder on the device to add the scripts to the selection, normally your [SYSROOT]/sbin folder).
Three buttons (start, stop and status) are predefined for start-stop-status scripts, there is also a parameter input field for more start values.
Free GPL v3 source code also included (Eclipse project). Please don't expect too much eye-candy. The GUI could be much better, but my CSS skills are very limited...
I added a few example scripts in the zip file, for example a battery stats reader. Before starting these scripts, have a look in the source code to be sure that they are compatible with your device.
Update 0.97 02/03/2012: Now the stderr is also printed in the output window
Update 0.99 02/10/2012: Replaced scrolling api with a much better solution (jScrollPane) for the output window. Not the real Android scroll feeling, but it's usable.
Attached Files
File Type: zip scriptstarter-v0.99.zip - [Click for QR Code] (610.3 KB, 801 views)
File Type: zip scriptset-2.6.zip - [Click for QR Code] (355.5 KB, 980 views)
The Following 66 Users Say Thank You to smitna For This Useful Post: [ Click to Expand ]
 
jukyO
Old
#2  
jukyO's Avatar
Senior Member
Thanks Meter 271
Posts: 1,513
Join Date: Feb 2011
Location: Ogulin
Thanks! I would hit the Thanks button, but I am on my mobile atm.

Sent from MIUI powered phone
||HTC Desire|| Nikez-Evervolv-Turba 3.3.0p1 (JB 4.2.2)|| 32.56.00.32U_5.17.05.23|| HTC-RIL 1.6.1062G||
2.6.38.8-evervolv-turba-jellybean-nikez-perfection||1Gb Ext4+14.4Gb FAT32


GUIDES
ADB Workshop and Guide for everyone
Installation guide to ADB on Ubuntu 10.10

MY Stuff
Juwe RAMscript Flashable zip
LG Keyboard
Bravia engine for Desire
CM7&MIUI sdcard read/write fix
 
smitna
Old
#3  
Senior Member - OP
Thanks Meter 110
Posts: 161
Join Date: Aug 2011

 
DONATE TO ME
Someone tried it? Problems?
Please, I need your feedback...
 
ukanth
Old
#4  
Recognized Developer
Thanks Meter 1220
Posts: 684
Join Date: Nov 2010

 
DONATE TO ME
Sounds interesting, I'll try once I get back home and share feedback here

AFWall+ Android Firewall
Save Energy & Save Earth - MonitorES
Contact me - Google+ Twitter
Github: Profile


 
dateno1
Old
(Last edited by dateno1; 18th January 2012 at 12:20 PM.)
#5  
Senior Member
Thanks Meter 42
Posts: 214
Join Date: Oct 2010
Default now i try to all packages for test

i will test on cortex a8 (motolora xt720)

TARGET_MARCH="armv7-a"
(not work on armv7 )
TARGET_MTUNE="cortex-a8"
TARGET_MFPU="neon"
TARGET_MFLOAT="softfp"


it make dynamic binarys but i don't want dymanic type (because it difficult to divide one thing and maybe not work on android's libs)

how can i make static type?


ps : oh...it has so many comile error (even can't read or count all)
 
smitna
Old
(Last edited by smitna; 18th January 2012 at 12:01 PM.)
#6  
Senior Member - OP
Thanks Meter 110
Posts: 161
Join Date: Aug 2011

 
DONATE TO ME
Quote:
Originally Posted by dateno1 View Post
i will test on cortex a8 (motolora xt720)

TARGET_MARCH="armv7-a"
(not work on armv7 )
TARGET_MTUNE="cortex-a8"
TARGET_MFPU="neon"
TARGET_MFLOAT="softfp"


it make dynamic binarys but i don't want dymanic type (because it difficult to divide one thing and maybe not work on android's libs)
I hope I understand your question. Why not dynamic?
Yes, you have to set the parameters for your mobile phone. For TARGET_MARCH try "armv7", for TARGET_MFLOAT "softfp" and for TARGET_MFPU "vfp" but check this last setting carefully for this phone.
It's very hard to build a full static binary (and sometimes nearly impossible) and it's also hard to build against bionic (with NDK). I had no good experience with it . A lot of compiler runs wasn't sucessful. Edit: I also had for example massive problems with openssl. I couldn't find a way to build it with NDK or static and then build other programs like cURL with openssl support.
Static binaries are also too big and not necessary. For this we have a sysroot environment to be independent from bionic and without the restrictions. The compiled libs and binaries are linked to the CodeSourcery glibc and have set the custom "rpath" for the dynamic loader. That's no problem I think.
 
dateno1
Old
#7  
Senior Member
Thanks Meter 42
Posts: 214
Join Date: Oct 2010
that's simple

if someone want to make some binary for all version of android (even same cpu) it will not work well (defend on basic lib(ex : libc) is ok but complex things will not work)

i know make static type binary is very difficult but it valable
 
smitna
Old
#8  
Senior Member - OP
Thanks Meter 110
Posts: 161
Join Date: Aug 2011

 
DONATE TO ME
Quote:
Originally Posted by dateno1 View Post
that's simple

if someone want to make some binary for all version of android (even same cpu) it will not work well (defend on basic lib(ex : libc) is ok but complex things will not work)

i know make static type binary is very difficult but it valable
You are right, if you want to build a "portable" binary, you have to set very safe settings, like TARGET_MARCH="armv5", the full list is in the "README". But it's not my main target to offer a solution that is easy portable to other phones, instead you can build the software with optimizations for your phone to "get the best" for it
 
dateno1
Old
#9  
Senior Member
Thanks Meter 42
Posts: 214
Join Date: Oct 2010
Quote:
Originally Posted by smitna View Post
You are right, if you want to build a "portable" binary, you have to set very safe settings, like TARGET_MARCH="armv5", the full list is in the "README". But it's not my main target to offer a solution that is easy portable to other phones, instead you can build the software with optimizations for your phone to "get the best" for it
just i want to add some more function on the phone

thanks for patch files
 
dancer_69
Old
#10  
Senior Member
Thanks Meter 191
Posts: 1,813
Join Date: Jan 2007
I want to try this, to install some apps(most interested fro mc), but I don' t wanna change my sd card.
Can I use phone' s /data partition as mysysroot directory? I have 1.5GB space available on data partition, it is 2GB all.
And If yes, I must create the dir: /data/mysyroot? And where to create this dir on pc? On root directory of my linux system?

Tags
android, cross compiling, linux installer, native apps, script, shell
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes