[SCRIPTSET v2.6][02/28/2012] Build native GNU/Linux applications: the easy way

Search This thread

smitna

Senior Member
Aug 6, 2011
161
116
i tried 843 port

i think it was android's libc.so's problem :(

when i trying python version it print out that message

No, that's not a libc problem. I tried it and the server is running:

./flashpolicyd.rb --xml flashpolicy.xml --logfile flashpolicyd.log

Output:
# Logfile created on 2012-02-03 17:16:18 +0000 by logger.rb/31641
I, [2012-02-03T17:16:18.708175 #23151] INFO -- : Starting server on port 843 in process 23151

netstat -an | grep 843

tcp 0 0 0.0.0.0:843 0.0.0.0:* LISTEN



Apart from this I saw that lightIRC is freeware, not Open Source.
So I will not include this program in the script code, sorry.
If you want to use it, you have to add it yourself.
 
Last edited:

dateno1

Senior Member
Oct 15, 2010
217
45
No, that's not a libc problem. I tried it and the server is running:

Code:
./flashpolicyd.rb --xml flashpolicy.xml --logfile flashpolicyd.log

Output:
# Logfile created on 2012-02-03 17:16:18 +0000 by logger.rb/31641
I, [2012-02-03T17:16:18.708175 #23151]  INFO -- : Starting server on port 843 in process 23151

netstat -an | grep 843

tcp        0      0 0.0.0.0:843             0.0.0.0:*               LISTEN


Apart from this I saw that lightIRC is freeware, not Open Source.
So I will not include this program in the script code, sorry.
If you want to use it, you have to add it yourself.

ok i try it again

i don't want to add lightirc just want to add ruby if you can (i think it is more valuable)

if you add ruby on your package i just download rb file and run that :D

---------- Post added at 03:22 AM ---------- Previous post was at 03:08 AM ----------

it my mistake :(

original ruby install on /usr/bin and /usr/lib

but i think it load it's all library from LD_LIBRARY_PATH but it can't recognize LD_LIBRARY_PATH

ruby just read from /usr/lib not LD_LIBRARY_PATH :(

now i make symlink from /system/lib/ruby to /usr/lib/ruby and it work well :)
 

Attachments

  • sc.jpg
    sc.jpg
    104.8 KB · Views: 192
  • sc2.jpg
    sc2.jpg
    186.2 KB · Views: 191
Last edited:

dateno1

Senior Member
Oct 15, 2010
217
45
screenshots of my phone

http://dateno1.egloos.com/1100488
sorry korean only but you can see the screenshots (so many screenshot i can't direct upload here)

all server and interpreter run on my phone and remote connected by ADB :)

special thanks for smitna ;)
 

smitna

Senior Member
Aug 6, 2011
161
116
new version 2.2 on first page

A new version 2.2 is now released.
A lot of internal changes, more features and updates, see page 1 and README for details.
Added Ruby and dosfstools

more to come...

Please report me potential bugs and also suggestions.
 

dateno1

Senior Member
Oct 15, 2010
217
45
this is

my make binary (static compiled)

it already tested (i was make shc and other things by it)

i think it better then new compiled thing (reduce time, work space, etc...)
 

Attachments

  • make.rar
    511.2 KB · Views: 22
Last edited:

smitna

Senior Member
Aug 6, 2011
161
116
Added SANE (USB Scanner backend and drivers) and more

Released version 2.5
Now it's possible to connect an USB scanner directly to the device and scan as usual. Additional with lighttpd and PHP it's possible to use the phpSANE frontend:
 

Attachments

  • phpsane-1.jpg
    phpsane-1.jpg
    42.1 KB · Views: 812
  • phpsane-2.jpg
    phpsane-2.jpg
    117.6 KB · Views: 603

smitna

Senior Member
Aug 6, 2011
161
116
Added Cuneiform OCR and tesseract-ocr

Released version 2.6
Two OCR tools added, see README for details (tesseract-ocr). The recognition quality with tesseract-ocr brings a surprisingly good result :) and in combination with SANE an useful addition to work with scanned text documents (from a scanner or from the camera).
 

ngh55

Senior Member
Dec 8, 2007
270
29
Did I read QUEMU??

Will this load an existent distro inside my beloved android phone?
 

smitna

Senior Member
Aug 6, 2011
161
116
Did I read QUEMU??

Will this load an existent distro inside my beloved android phone?

It's a full featured version of QEMU. I have not tried to boot a complete distro, only some i386 programs, but yes, it should work. Don't expect high performance.
 
Last edited:

enlight22

Member
Feb 7, 2009
22
0
first may i say... this script is total awesomeness, thanks to the author..

I have used it to sucessfully compile DnsCrypt-proxy for android... and its working.

It should perhaps be added to the package list

I do have a question though.. i was unable to find the propper settings for my phone... so i used the ultra safe settings the readme suggests.. armv5 and whatnot.

It seems the program works fine, and its a pretty simple program it seems.. Is there any reason to figure out the settings for my phone, which i know to be armv7 cortex a8, but i dont know the other settings.
what would the benefit of compile with the propper settings be?

also... do i really need all that crap that is compiled by default? i dont really know why i would need... python, or openssl for example.. and the build readme for this dnscrypt program doesnt specify and dependencies..

it is the only program i need and is only a few hundred k.. i dont see why i need this 80 or so mb of other stuff in my /data/mysysroot folder.
 

smitna

Senior Member
Aug 6, 2011
161
116
first may i say... this script is total awesomeness, thanks to the author..

I have used it to sucessfully compile DnsCrypt-proxy for android... and its working.

It should perhaps be added to the package list

I do have a question though.. i was unable to find the propper settings for my phone... so i used the ultra safe settings the readme suggests.. armv5 and whatnot.

It seems the program works fine, and its a pretty simple program it seems.. Is there any reason to figure out the settings for my phone, which i know to be armv7 cortex a8, but i dont know the other settings.
what would the benefit of compile with the propper settings be?

also... do i really need all that crap that is compiled by default? i dont really know why i would need... python, or openssl for example.. and the build readme for this dnscrypt program doesnt specify and dependencies..

it is the only program i need and is only a few hundred k.. i dont see why i need this 80 or so mb of other stuff in my /data/mysysroot folder.

Thanks for your post and feedback, I will add DnsCrypt-proxy in the next release.

The first step to check the proper setting for your phone is to open a terminal and type
cat /proc/cpuinfo
It will show you the arch/processor generation and the features (thumb/neon etc.). The benefit to use the "best" settings is a faster execution and other optimisations of the compiled binary. But I don't think that it's necessary for DnsCrypt-proxy to spend too much time to find all the right compiler switches. It's not a program for time critical usage like a packer, converter etc. If you want to compile other applications with the help of my script like ImageMagick or some script languages then you should of course use the best settings for your phone.
Just use the "safe settings" or take the settings for an ARM v7 generation phone (armv7-a, cortex-a8). You can see the full list with the possible options with
arm-none-linux-gnueabi-gcc --target-help
in the arm-2011.09/bin folder (CodeSourcery)

You don't need to compile any other packages from "ALWAYS NEEDED" if you are sure that there are no further dependencies. Simply set them to "no".
My goal was to exclude as much possible compile errors. Most of the packages in the script set needs the openssl, zlib and ncurses libraries to work.
But you need in any case the libraries (cs-sysroot.tar.bz2) because nearly all generated binaries use these libraries and not the standard libraries of the phone (bionic glibc etc.) and you must copy the files for the first time to your phone. The only way to circumvent this is to use the Android NDK (for bionic). Have a look in the build.sh script, section for bash. But this can lead in a LOT OF WORK to make the sources compatible with NDK to sucessful build the binaries :mad: and I can't recommend this.
 

mai77

Senior Member
Nov 16, 2011
1,429
580
bbsh

is it not easier to use botbrew or TerminalIDE :confused:

afaik u can achieve the same goals there with bash and bbsh
 

nyarnon

Senior Member
Apr 29, 2007
324
66
This is ubercool. With the new devices like MK802 the command line will get important again. As soon as I see fit I will put up binaries for it. Kudossmitna.

Sent from my GT-I9000 using xda premium
 

nyarnon

Senior Member
Apr 29, 2007
324
66
Well that was to be expected, just to easy. Compiler stops at the first package NCurses, looks like I need a patches autoconf2.13 to make it happen.


checking for C compiler default output... configure: error: C compiler cannot create executables
sed: kan ./include/ncurses_cfg.h niet lezen: Bestand of map bestaat niet

checking for the file reveales ncurses_cfg.hin with the text

This is a template-file used to generate the "ncurses_cfg.h" file.
*
* Rather than list every definition, the configuration script substitutes the
* definitions that it finds using 'sed'. You need a patch (original date
* 971222) to autoconf 2.12 or 2.13 to do this.
*
* See:
* http://invisible-island.net/autoconf/
* ftp://invisible-island.net/autoconf/
 

smitna

Senior Member
Aug 6, 2011
161
116
Hi.

I didn't checked all the applications, but do you have redsocks ? If not can you add it ?
Thanks in advance.

http://darkk.net.ru/redsocks/
https://github.com/darkk/redsocks

Sorry for my late answer. I am very busy at the moment. But I am sure it's possible to add redsocks (I saw that ProxyDroid and sshtunnel use redsocks). Why don't you want to try it yourself? The script gives you all the ability (with help from the docus) to add the tool. But I add it to my "todo list".
 

Top Liked Posts

  • There are no posts matching your filters.
  • 70
    New: SANE (USB Scanner backend and drivers) - OTG equipped device needed to work for this - YES, Scanner Access Now possible and Easy with your phone :)

    This can (in combination with tesseract-ocr) be very useful to recognize text from scanned images.

    For pictures see also http://xdaforums.com/showpost.php?p=23046120&postcount=187

    Update 01/30/2012: Now with a small android app (ScriptStarter) to start services and other shell scripts easily from the GUI. See bottom of the post.


    WHAT'S POSSIBLE WITH THIS SCRIPT:

    With this bash script your are able to build relatively easy from source a lot of native software (over 90 at the moment) from the GNU/Linux community for your Android phone and you will have
    the good experience with free software.
    All these tools and programs are not "apps for the GUI" for Android but native running command line tools "under the surface" of your Android GUI like the kernel and other services.
    Don't ask me if it makes any sense to install program xy (like MySQL) on a phone. This is your decision. It's possible, so try it and see what you can do with it.

    Note: I tested most of the compiled software on a Samsung Galaxy S2 (my phone). I cannot test it on a variety of phones because I have no mobile phone shop...


    Currently buildable with this scripts (abstract, see the full list in package_selection.conf):

    • PHP 5.3.10 (with almost all extensions and XCache)
    • MySQL 5.5.19 (if you want a full database server on your phone...)
    • lighttpd 1.4.30
    • OpenSSH 5.9p1
    • OpenSSL 1.0.0f
    • Samba 3.6.3
    • cURL 7.23.1
    • bash 4.2
    • BusyBox 1.19.3
    • ImageMagick 6.7.4-10
    • OpenVPN 2.2.2
    • Python 2.7.2
    • Node.js 0.6.7
    • git 1.7.9
    • Wget 1.13.4
    • Rsync 3.0.9
    • TCPDUMP 4.2.0
    • Midnight Commander 4.8.1
    • Pure-FTPd 1.0.35
    • screen 4.0.3
    • iptables 1.4.10 (without bionic bug "getnetbyaddr()" and with iptables-save/restore!)
    • BIND 9.8.1-P1
    • Lua 5.2.0
    • Ruby 1.9.3-p0
    • usbutils 004
    • MTDUtils latest git
    • GCC 4.6.2 (c, c++)
    • GNU Coreutils 8.14, Binutils 2.22, Findutils 4.4.2, Inetutils 1.8
    • QEMU 0.15.1
    • SANE 1.0.22
    • tesseract-ocr 3.01
    • ...
    • a lot of libraries, development tools and other useful programs
      ...


    FEATURES:

    • automatic download and untar the cross-compilers (if not already done) for cross compiling
    • automatic download and untar the source packages (if not already done), that you have selected
    • solve dependences and reserves additional needed packages
    • apply patches to the sources if necessary
    • build them, "install" them in a folder for later "ready to transfer" to your phone and stripes the binaries and libraries to reduce the filesize if selected
    • autogenerate a tar file and adb push the file to device if selected
    • allows you to use well-known software like PHP, MySQL, OpenSSL, OpenSSH, OpenVPN, bash and lighttpd on your phone (see the full list in file package_selection.conf, see attachement)
    • scripting and development on your phone
    • no limits with BusyBox (without want to be derogative, BusyBox is a wonderful tool), full featured tools like Coreutils, Binutils etc.
    • build scripts are expandable with other or newer packages you eventual want to have

    QUESTIONS AND ANSWERS:

    Q: Do I need native GNU/Linux programs and what is that?
    A: If you do not know what native Linux applications are, then stop here - you don't need them.

    Q: What can I do with these tools?
    A: Many things. As a developer you can use the libraries to include them in Android GUI apps, as user you can start for example a webserver on your phone and write some HTML or PHP pages
    just for fun or replace BusyBox with full featured tools (or only some commands).
    Some tools are also very useful to find network errors, check the network traffic and many more.

    Q: What do I need?
    A: A Linux installation (VM or real), some Linux knowledge and a little bit time to read all instructions carefully. A rooted Android device with enouth free memory (ext2/3/4 formatted)
    See README for details.

    Q: Costs?
    A: Nothing. The scripts are Open Source (GPL v3), the packages are GPL licensed or under a similar license. The cross-compilers are Open Source or free to use for non-profit purposes.
    But I would be pleased if I get a donation and/or a press on the "Thanks button", if the scripts are useful for you. It was a long and hard work...


    Note: The attachement scriptset-x.yy.zip is double packed (inside .bz2), because I want preserve the right settings of the scripts.

    STEPS:
    1. Selected the desired packages in the config file
    2. Adjust the compiler settings for your phone
    3. Run the build script to download and compile the software
    4. Copy the generated TAR files to your phone or let "adb" this do for you and unpack them in the prepared sysroot folder

    For more information please read the instructions in "README"
    Please let me also know if you have any problems or if you have questions or further suggestions.

    Thanks go to the Open Source community and a lot of people/webpages with hints about cross-compiling software.

    Thanks

    smitna

    CHANGELOG:
    01/14/2012 First public release v0.9
    01/14/2012 Update: Removed unneeded big patch for ncurses
    01/15/2012 Release of v1.0
    Update: Fixed build of gcc. Note: the header files are not copied automatically with the script.
    If you want to compile on your phone, set TAR_HEADERS=1 in configuraton.conf
    Typos: some translations corrected in build.sh
    01/16/2012 Release of v1.1
    Update: Removed building of fsck from util-linux-ng, it's in e2fsprogs
    Update: ISC Cron / vixie cron: set correct paths in pathnames.h before compiling
    Enhancement: More detailed explanation of device specific compiler settings with list of possible values
    Update: Removed dependence e2fsprogs libs from e2fsprogs
    01/17/2012 Release of v1.2
    Update: Build of zip, zipcloak, zipnote, zipsplit, unzip, unzipsfx and funzip now possible without errors
    01/17/2012 Release of v1.3
    Update: Added Node.js
    Typos: some translations and language errors corrected in build.sh and README (sorry, I'm not a native English speaker...)
    01/18/2012 Release of v1.4
    Fixed TMP_PATH in PHP extension XCache (mmap.c) to prevent startup warning
    Fixed BusyBox problems with static build against glibc ("__getpagesize" segfault with ping, traceroute and other networking parts)
    and added some BusyBox patches.
    BusyBox is now dynamic linked, so don't replace the system BusyBox with this build, because of missing mount/paths to libraries at phone startup
    01/20/2012 Release of v1.5
    Update: Cleanup in scripts and separation of configuration and code. Now it's only necessary to edit configuration.conf and package_selection.conf
    Update: Added gzip
    Update: Added experimental adb push to device if the device is connected and permissions are ok
    Update: Check for installed programs. Script now exists on check failure to minimize errors while compiling.
    01/21/2012 Release of v1.6
    Bugfix: MySQL build could fail if gen_lex_hash could not generate the file sql/lex_hash.h in pass 2
    Update: adb push should now work, check for connected device (and possible multiple devices) and check for right permissions to use the device
    Enhancement: Even more explanation of device specific compiler flags and some hints to find the right settings.
    01/21/2012 Release of v1.7
    Enhancement: To eliminate sure a failing MySQL build, with every new run the source folder will be removed before completely
    Enhancement: file version is now strictly checked (the host version must be the same as the cross version)
    Some minor changes and filechecks (e.g. autoconf)
    01/23/2012 Release of v1.8
    Bugfix: Python has SQLite dependencies and SQLite must be ready before Python configure/make
    Bugfix: In certain circumstances the optimizer settings get lost
    Bugfix: Removed --enable-multibyte from bash, no chance to use it with NDK
    01/24/2012 Release of v1.9
    Update: Added ngIRCd (Please read the comment for this server, because of running as root)
    Update: Sorted the links and package names in the config a little bit...
    01/26/2012 Release of v2.0
    Bugfix: OpenSSH wants to touch /var/empty while compiling, should be fixed --with-privsep-path now set (thanks to member dateno1 for his reports of such warnings/errors)
    Bugfix: Removed a wrong CFLAG -I path from NDK config (only useful for me, really useless for others...)
    Enhancement: Example for a user db for Pure-FTPd virtual user database in README (please use virtual users, it's the best and easiest solution)
    Update: OpenSSH build need ssh-keygen for the generation of the default keys, added this info to README and the package selection
    Bugfix: Not really a bugfix, but some installers don't set the right permissions of libraries in the sysroot folder, so set them global in the final task
    Bugfix: Removed --with-lastlog=$TARGET_SYSROOT/var/log/lastlog from OpenSSH and added --with-lastlog=no, I don't know why this is not working and throws a warning, but we can live without it...
    Enhancement: Added option to choose the sysroot variant from the CodeSourcery libs: v5TE (default), v4T (old), thumb2 (v7, fastest if supported by the phone)
    Enhancement: Added option to tar the headers for development on the device with CodeSourcery (hint: always set rpath in LDFLAGS to make use of the CodeSourcery libs)
    01/26/2012 Release of v2.1
    Update: Added getcap/setcap and libraries from wishlist, needs libtoolize. Note: This is in a testing stage, kernel and filesystem support needed!
    01/30/2012 Added a ScriptStarter app
    09/02/2012 Release of v2.2
    Update: Local compilation of static "tic" from ncurses for buildhost, if ncurses installation is missing
    Enhancement: "downloading only" mode without compiling the packages (and force download)
    Enhancement: some parameters for starting the script with additional functions (show with ./build.sh help)
    Enhancement: more logging and outputs
    Enhancement: language specs, all outputs can be changed to other languages in build.sh
    Update: Added dosfstools
    Update: Added Ruby
    Update: Added example configs and starter scripts for Samba and OpenSSH
    Update: Use of http-URLs if possible and update of some broken external links since last release
    Bugfix: util-linux-ng needs gettext, added dependence
    Bugfix: Added parameter to Samba to solve user/group settings and socket_wrapper support to set path vars at runtime
    Update: Changed PHP version to 5.3.10, Samba to 3.6.3 and git to 1.7.9
    02/11/2012 internal Release v2.3
    Bugfix: openVPN patch for /dev/tun and links to ip/netstat/ifconfig/route, see hints for symlinks
    02/16/2012 internal Release v2.4
    02/25/2012 Release of v2.5
    Update: Added LAME
    Update: Added experimental CUPS
    Update: Changed PNG version to 1.5.9
    Update: Added Net-SNMP
    Update: Added Netpbm
    Update: Added PNG 1.2.47 for Netpbm and phpSANE
    Update: Added LibTIFF
    Update: Added SANE (USB Scanner backend and drivers) - OTG equipped device needed to work for this - YES, Scanner Access Now possible and Easy with your phone :)
    Update: Split patches for bash (4.1 and 4.2) for better and easier (automatic) inclusion of original patches
    02/28/2012 Release of v2.6
    Update: Added Cuneiform OCR
    Update: Added tesseract-ocr (see hints for more details howto use it)

    ScriptStarter
    02/03/2012 Added a small ScriptStarter for the GUI. With this app, written with JQuery, PhoneGap and Java (wrapper to execute the shell scripts) it's possible to start for example the lighttpd web server, Pure-FTPd or MySQL from within the Android GUI without a terminal and read the output. If a script needs root access, there is a checkbox to allow this. For this to work, copy the script sush in the script folder. All scripts ending with .sh can be started (please choose after the first start your script folder on the device to add the scripts to the selection, normally your [SYSROOT]/sbin folder).
    Three buttons (start, stop and status) are predefined for start-stop-status scripts, there is also a parameter input field for more start values.
    Free GPL v3 source code also included (Eclipse project). Please don't expect too much eye-candy. The GUI could be much better, but my CSS skills are very limited...
    I added a few example scripts in the zip file, for example a battery stats reader. Before starting these scripts, have a look in the source code to be sure that they are compatible with your device.
    Update 0.97 02/03/2012: Now the stderr is also printed in the output window
    Update 0.99 02/10/2012: Replaced scrolling api with a much better solution (jScrollPane) for the output window. Not the real Android scroll feeling, but it's usable.
    2
    One of the best threads of 2012. Great job documenting this :)

    Thanks
    1
    package selection : mysql only

    target directory (prefix) : /sddata/mysql (on sdext on my system)
    armv7-a
    cortex-a8
    neon
    softfp


    then run ./build.sh 2> error.txt

    it is my error messages

    http://pastebin.com/g7Yi4bGz
    sorry you can see this link only today (my mistake)

    this setting can make some binary and lib run on my phone but fail to make gcc or mysql :( (when i try gcc it just fail to making gcc only (other things were made) )


    Please notice first, that you should not run this script as root!!

    Then there is something wrong with the build of the first needed packages:
    ZLIB=yes
    OPENSSL=yes
    CURL=yes
    GETTEXT=yes
    LIBXML2=yes
    READLINE=yes
    NCURSES=yes

    - The script cannot find the readline sources. Do you have a subfolder readline-6.2 in folder sources in your builddir? Same for the other source folders. The script first downloads the packages to folder packages, extract the tars in folder sources, repeated for all selections.
    (look at line 67 in your log: ./build.sh: 1402: cd: Python-2.7.2: No such file or directory)
    - You have no "xz", install it first and make sure, that the other necessary packages are installed. MySQL and gcc are "two pass" compilations and this builds generating temporary binaries, runable on your linux pc. So you need all for a successful compilation of such binaries like gcc, make, python etc. on your linux machine. If you are using Ubuntu, then install the "meta-package" build-essential, look in the README.
    Do you have the other unpacking tools like gunzip etc. Please install also "bison"

    You should do following steps:
    1. First select only the necessary packages (see above), all others to no.
    2. If zlib, python, readline an the others are cross-compiled (without errors) and the libraries are in ./sysroot/lib, then deselect them (=no) and select mysql=yes and start a second run with ./build.sh
    1
    with this error

    note: the mangling of 'va_list' has changed in GCC 4.4

    full error log on http://pastebin.com/jd1Pw9Xw

    Hello dateno1,

    first: you have *NOT* changed to a user account as promised :mad:

    1. The va_list is only a note, not an error, no problem (but ask the MySQL devs why they haven't adapted the code to gcc 4.6.1).
    2. Your Python cross binaries and libraries are not ok (from you broken first build?). Please start from a "fresh state" (delete the folder sources/Python-2.7.2 and sysroot) or start from a total clean state.
    3. Please use the latest 1.5 version of the script because of the xargs warnings at the end
    4. Delete the complete sysroot folder, there are some folders not in place (data and docs). There is a script remove_all_files.sh
      Start this script in your buildroot with ./remove_all_files.sh to remove all files and symlinks in subfolder sysroot.
      Edit: delete the sysroot folder complete, there are some other missing folders from your first build!
    5. Start again with ./build.sh and activated packages (the "ALWAYS NEEDED" one and MySQL)
    1
    One of the best threads of 2012. Great job documenting this :)

    Thanks

    Yes, only my english should be polished :eek: