[Scripting][ARM][Installer][Port] Python (static) 2.7.9 and 3.4.2

Search This thread

7175

Senior Member
Feb 6, 2013
309
484
(Check out https://www.python.org for information on what python is.)


Yes you read that correctly, static python for Android! It took a while to figure out how to get this compiled, but I finally did it. I present you with an installer and a little bit of testing. The second post contains information on compilation and the sources. Btw, any cool python scripts are welcomed!


INSTALLATION
Recovery flash installers are attached that will install python 2.7.9 and/or 3.4.2 to /system/pythonX.X.X. You can install both if you want. Installation size is about 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.

After installation, the python director(ies) in /system will contain the static python binary, some scripts, and a bunch of modules and documents. Separate scripted executables will be installed to /system/bin/python or /system/bin/python3 depending on which one is installed. These basically just set the PYTHONHOME environment variable and execute the python binary.


TESTING
When you boot up to Android after installation, you should be able to just open up a terminal and fire off some python commands.

Test using python 2.7.9:
Code:
python -c 'print "Hello World!"'


Test using python 3.4.2:
Code:
python3 -c 'print("Hello World!")'


You can also write scripts shelled with python:

Code:
#!/system/bin/python

print "Hello World!"

Make sure to set them as executable with "chmod +x".


Adjust your screen brightness via /sys (root required, tested on Galaxy Nexus):
Code:
python -c 'f=open("/sys/devices/omapdss/display0/backlight/s6e8aa0/brightness","w"); f.write("40"); f.close()'

A cool script I made to tweak file system I/O like rq_affinity, rotational, etc for I/O blocks:
Code:
#!/system/bin/python

import os,re,sys

list=[]
# find all directories containing rq_affinity
for roots, dirs, files in os.walk('/sys'):
  for file in files:
    match=re.search(r'\S+/rq_affinity',os.path.join(roots,file))
    if match:
      list.append(match.group().replace('rq_affinity',''))

# write specific values to files in each directory found before
for dir in list:
  for name in 'rq_affinity', 'rotational', 'read_ahead_kb', 'nr_requests', 'iostats', 'nomerges', 'add_random':
    try:
      f=open(dir+name,'w')
      if name is 'rq_affinity': f.write('1')
      elif name is 'read_ahead_kb': f.write('512')
      elif name is 'nr_requests': f.write('512')
      else: f.write('0')
      f.close()
    except IOError:
      sys.stderr.write('Problem writing to ' + dir+name + '\n')

Or execute "python" or "python3" without any parameters to open up the interpreter and go from there. When you start for example "python" (python 2.7.8), you should see something like:

Code:
Python 2.7.8 (default, Dec  2 2014, 05:15:18) 
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>


***I haven't fully tested this yet, so let me know how it goes. I know basic commands work, but there are still some complexities in the compilation that may need to be figured out.***


(update) -- Install Extra Packages/Modules via "easy_install" -- (root required)
1) In a shell, remount /system read-writable with:
Code:
mount -o remount,rw /system
2) Change directory to /system/pythonX.X.X, i.e.,
Code:
cd /system/pythonX.X.X
3) Execute easy_install followed by a package name, i.e.,
Code:
./easy_install [i]package_name[/i]
4) Let it download and install! :) Then test it out.
5) Remount /system read-only with:
Code:
mount -o remount,ro /system

Also, to remove a package, execute:
Code:
./easy_install -m [i]package_name[/i]
rm -r ../lib/pythonX.X/site-packages/[i]package_name[/i]*.egg

*** Flashing the resolv.conf patch may be required to make an internet connection to download modules.

Also, a few things need to be done to get easy_install working:
1) For python 2.7.8 or 2.7.9 you need to link python2.7 to python like this:
Code:
ln -s /system/python2.7.9/bin/python2.7 /system/python2.7.9/bin/python
I'll make sure this is automatically done in future installs.

2) SSL certificates need to be added to "/etc/pki/tls/certs/" to use SSL with easy_install.
Code:
mkdir -p /etc/pki/tls/certs
curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt


ISSUES
The python installations have the following modules compiled in with external dependencies such as libraries not included at this point:
Code:
--------------PYTHON 2.7.8-2.7.9----------------------
_hashlib        _multiprocessing     _ssl             
_testcapi       bz2                   crypt
dbm
Code:
--------------PYTHON 3.4.2-----------------------------
 _crypt             _dbm                 _decimal           
_hashlib            _multiprocessing     _ssl
_testcapi
I will look into this more.

As @bubbleguuum points out, name resolution with python 3.4.2's urllib wasn't working. I found a workaround by adding the line
Code:
options single-request-reopen
to /system/etc/resolv.conf. This still needs some testing but seemed to work for me. I've included a recovery flash zip to patch /system/etc/resolv.conf if the line is not there since this is a root operation.

UPDATES
* 2014-12-02 * Compiled in more modules for each installation. This makes the installation a bit bigger, but it's worth it. You get a more complete python! :) To get stuff like help functions and math and readline modules, you need the larger installations linked below under "DOWNLOADS".
* 2014-12-03 * Thanks @cybojenix for pointing out some incapatibility issues with the original edify installer I was using. I have now updated the installers to use SuperSU's great non-edify sh-script installer.
* 2014-12-05 * Some updates to the installers to include more modules, plus size reduction for python 3.4.2. Modules added to python 2.7.8 : _bsddb _ctypes, _ctypes_test, _hotshot, _json, _lsprof, _sqlite3, future_builtins, and ossaudiodev, easy_install, pip(broken for now). Modules added to python 3.4.2: _bz2, _ctypes, _ctypes_test, _gdbm, _json, _lsprof, _opcode, _sqlite3, _testbuffer, _testimportmultiple, ossaudiodev, xxlimited
* 2014-12-05 * Some big updates to the installer to include almost the rest of the modules that wouldn't compile. Also easy_install is working on my end with these new experimental builds. With a little effort, should be able to get pip working as well. I had to use some hacky flags to get it to compile, ignore unresolved-symbols and such, so it definitely needs some testing. Another thing I did was tweak the install script to write over previous installations from these installations. I recommend downloading the experimental builds if you read this. The worst thing that might happen is one of the new modules I've included won't work completely.
* 2014-12-14 * Added python 2.7.9. Trimmed the installation sizes down to the much more reasonable 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
* 2014-12-24 * Attached a recovery flash.zip to patch resolv.conf to try and fix a urllib name resolution error.
* 2014-12-26 * A couple updates: Tweaked python installations so that modules looks for "/system/bin/sh" instead of "/bin/sh" when needed. Also a few "#!" corrections for scripts in the python bin directory. This shouldn't break anything from what I've tested, only make Android python more capable, but I'll keep the old installations attached for now just in case since the modifications involved using "sed". The second update is nice. I've added a little section on installing extra modules with easy_install(included and working with all installations).
 

Attachments

  • python2.7.8_noedify_installer.zip
    12.5 MB · Views: 2,164
  • python2.7.9_noedify_installer.zip
    14.1 MB · Views: 3,798
  • python3.4.2_noedify_installer.zip
    15.3 MB · Views: 3,486
  • resolv_ipv6_patch.zip
    4 KB · Views: 1,027
  • python2.7.8_noedify_installer_26dec2014.zip
    12.5 MB · Views: 692
  • python2.7.9_noedify_installer_26dec2014.zip
    14.1 MB · Views: 1,348
  • python3.4.2_noedify_installer_26dec2014.zip
    15.3 MB · Views: 3,038
  • curl.zip
    457.3 KB · Views: 1,345
Last edited:

7175

Senior Member
Feb 6, 2013
309
484
Compiling
The environment I compiled in was a debian "wheezy" image mounted as a loop device on Android. The reason I compile this way is so I have full access to the arm environment tools, gcc compiler, etc. without having to go through the trouble of using a cross-compiler (which doesn't work in this case from my testing with uClibc, or eglibc. Code sourcery's might work but I didn't feel like booting an x86 linux installation to try). You can recreate the same environment pretty easily by using either Complete Linux Installer or Linux Deploy apps. I use both.

To get a static python compiled, I first downloaded the following packages with APT:
Code:
[i]apt-get build-dep python python3[/i]
build-essential gcc make # building tools
binutils-gold # awesome gold linker
zlib1g-dev # zlib
libreadline-dev, libncurses5-dev # readline, curses
libbz2-dev # bz2
libsqlite3-dev # sqlite3
python-bsddb3, python3-bsddb3
libgdbm-dev # gdb
libssl-dev # ssl
python-tk, python3-tk # tkinter
libdb-dev # db
python-gdbm python-bsddb3
libffi-dev # _ctypes
tcl8.6-dev # tkinter
libx11-dev # tkinter
libmpdec-dev # decimal

Then I manually installed binutils-gold by unpacking the rpm for armv7h (this isn't available with APT yet). You can get it here though. (Update: this should be available in the debian repositories now.)


For static compilation, you need to make some changes to Modules/Setup from the python source directory. Specifically, you need to add
Code:
*static*
to the top of the file. Then you need to uncomment any commented modules you would like compiled in. When you compile and see a list of failed modules, search for each of these in Modules/Setup and uncomment them. For example, change
Code:
#math mathmodule.c _math.c
to
Code:
math mathmodule.c _math.c
. Then recompile. There may be some shared dependencies and packages as well you need to figure out to get them to compile. Take a look at the attached "Setup" files for more detail.



The actually compile steps look like this:
(1)
Code:
./configure --build=arm --prefix="$PWD"/out LDFLAGS="-static -static-libgcc -Wl,--unresolved-symbols=ignore-all -Wl,--export-dynamic" CPPFLAGS=-static CXXFLAGS=-static CFLAGS="-Os -static" LDFLAGS=-static LD=ld.gold
(2) Modify the Modules/Setup file.
(3)
Code:
make clean; make install



After compiling, there are some things you can do to make your installation in "./out" smaller. Strip the large binaries in "out/bin", i.e.
Code:
strip -s ./out/bin/python2.7
Also remove the python archive library, *.o's, etc. with:
Code:
make clean
This might seem kind of weird, but it works and cleans out unnecessary files from your installation.



**Make sure to backup your Modules/Setup file if you run "make distclean". I lost my 3.4.2 Modules/Setup file after doing this. :silly: Now I gotta recreate it, dammit.
**I should also mention that compile time is very fast given the amount of data generated. It takes about 10 minutes to get python and all it's modules compiled on my Galaxy Nexus.**

:EDITS:
: Updated some configure parameters. Removed unnecessary, extra "./configure". Also added some updates to the included "Setup" files.
: Hacky update to "./conifgure" to ignore unresolved symbols, export dynamic. This allows certain modules to be compiled which have shared dependencies.
: Update to trimming down the installation size after "make". Use another "make clean".

Downloads - Extract in the source directory
Setup (python 2.7.8)
Setup (python 3.4.2)
Setup (Experimental) (python 2.7.8)
Setup (Experimental) (python 3.4.2)
 

Attachments

  • Python-2.7.8-source.tar.gz.zip
    14.3 MB · Views: 326
  • Python-3.4.2-source.tar.gz.zip
    18.5 MB · Views: 511
Last edited:

cybojenix

Retired Forum Mod / Retired Recognized Developer
Apr 7, 2012
801
1,864
Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.

Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.

Give me a shout if you need anything. I hope to see even more come out of this thread ;)

Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
 
Last edited:
  • Like
Reactions: 7175

7175

Senior Member
Feb 6, 2013
309
484
Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.

Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.

Give me a shout if you need anything. I hope to see even more come out of this thread ;)

Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
Thanks for your reply! I've changed the installer to use SuperSU's no-edify sh-scripted updater. I tested on my device, but let me know if there are any issues though. I kept it pretty basic.

I added some stuff on the source modifications and compilation to the 2nd post. The only file I've actually modified so far in the source is the generated Modules/Setup file, and I've included the one I used for python 2.7.8. Unfortunately, I over-cleaned the python 3.4.2 directory, which deleted my modified Modules/Setup there, so I'll have to add it later when I re-edit it.

That's a great idea on pip/easy_install. I did download the pip_installer and tried out installing on the static python I made. Got some errors though pointing to a few modules I haven't gotten compiled into the static python installation (listed in the OP near the bottom, specifically _ctypes). I'll see what I can do about that. The line for compiling the module might just need to be added to Modules/Setup. Hopefully that's the case.
 

bubbleguuum

Senior Member
Dec 23, 2010
7,016
2,745
@7175

Thank you very much for these binaries.

There is however a problem: name resolution (DNS) doesn't seem to work at all (with both python 2 and 3 downloads).
All attempts to use urllib.request.urlopen('http://somehost.com') fail with "<urlopen error [Errno -2] Name or service not known>" (running python as root but it doesn't matter, and a rooted Nexus 4 running 4.4.4 stock ROM):

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 455, in open
response = self._open(req, data)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 473, in _open
'_open', req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 433, in _call_chain
result = func(*args)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1202, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1176, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

it works if replacing somehost.com by its ip address.

To reproduce above stack trace with python 3:

import urllib.request
urllib.request.urlopen('http://www.google.com')

while this work:

urllib.request.urlopen('http://173.194.45.229')

nslookup and general name resolution work anywhere else:

root@mako:/ # nslookup www.google.com
nslookup wwwgoogle.com
Server: 8.8.4.4
Address 1: 8.8.4.4 google-public-dns-b.google.com

Name: wwwgoogle.com
Address 1: 2a00:1450:4007:80c::1014 par03s15-in-x14.1e100.net
Address 2: 173.194.45.242 par03s15-in-f18.1e100.net
Address 3: 173.194.45.240 par03s15-in-f16.1e100.net
Address 4: 173.194.45.244 par03s15-in-f20.1e100.net
Address 5: 173.194.45.241 par03s15-in-f17.1e100.net
Address 6: 173.194.45.243 par03s15-in-f19.1e100.net

This issue has probably something to do with how python was compiled to do name resolution. I remember vaguely a similar issue
on Ubuntu 12.04, with a ffmpeg compile that would always fail to resolve hostnames in URLs. No sure what the exact fix was but it had something to do with the libc and its name resolution mechanism. Or maybe something ipv6 related ?
 
Last edited:
  • Like
Reactions: 7175

7175

Senior Member
Feb 6, 2013
309
484
@bubbleguuum : Thanks for checking out that important issue.

I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.

EDIT: Hmm, looks like re-compiling with "--disable-ipv6" didn't fix the problem. Will have to look more into this issue and hold off on uploading ipv4-only python3 since it didn't fix it. In the meantime, I've attached a recovery flash.zip to the OP to add the above mentioned line to /system/etc/resolv.conf if needed, which seems to fix the issue for me.
 
Last edited:

bubbleguuum

Senior Member
Dec 23, 2010
7,016
2,745
@bubbleguuum : Thanks for checking out that important issue.

I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.

Thank you for looking into this so fast.

My intended usage is for running the python binary from a regular non-root app (which works great!), so editing resolv.conf is not an option.
 
  • Like
Reactions: theallegro

bubbleguuum

Senior Member
Dec 23, 2010
7,016
2,745
I've been researching that DNS resolving issue and could not find a clear explanation on why it fails on Android and not on other systems, and why exactly the added line in resolv.conf fixes (or rather workaround) it.

All seems to point to socket.getaddrinfo() failing for an unknown reason.

Is Python compiled with HAVE_GETADDRINFO defined ? If that's the case it uses the libc definition of getaddrinfo, otherwise
it uses an emulation function found in getaddrinfo.c.
My guess is that it is the latter, and what is causing this issue on Android.

It's probable this issue has already been solved on Android since other Python binaries exists, but Googling around do not give
much answer.
 

Hairo The Blue Mage

Senior Member
Jul 1, 2011
375
118
Moca
I get this error when trying to install anything with easy_install (using /system/python3.4.2/bin/easy_install-3.4 because /system/python3.4.2/easy_install doesn't seem to exist like in the OP):

Code:
# ./easy_install-3.4 feedparser
Searching for feedparser
Reading https://pypi.python.org/simple/feedparser/
Download error on https://pypi.python.org/simple/feedparser/: [Errno -2] Name or service not known -- Some packages may not be found!
Couldn't find index page for 'feedparser' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno -2] Name or service not known -- Some packages may not be found!
No local packages or download links found for feedparser
error: Could not find suitable distribution for Requirement.parse('feedparser')

i got curl and the certs installed in /etc/pki/tls/certs/ as per instrucions in the OP, any help?
 
Aug 17, 2013
46
99
Kyiv
Hmm... new problems here. Android Lollipop have no /system/etc/resolv.conf by-default and urllib patch is no more works.
How it can be fixed?
 

hackintosh5

Senior Member
Sep 15, 2017
1,246
957
Where I can find source code and related documents for project

I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?
 

Surge1223

Recognized Contributor
Nov 6, 2012
2,622
7,466
Florida
Google Pixel 6 Pro
I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?

Idk about OP, but I can surely post my source for python 2.7 on arm/arm64. I compiled it like a year ago, but it wasn't too bad, just required a bunch of patches and code for dlopen
 

Attachments

  • Screenshot_20171218-062028.png
    Screenshot_20171218-062028.png
    233.7 KB · Views: 266

Top Liked Posts

  • There are no posts matching your filters.
  • 17
    (Check out https://www.python.org for information on what python is.)


    Yes you read that correctly, static python for Android! It took a while to figure out how to get this compiled, but I finally did it. I present you with an installer and a little bit of testing. The second post contains information on compilation and the sources. Btw, any cool python scripts are welcomed!


    INSTALLATION
    Recovery flash installers are attached that will install python 2.7.9 and/or 3.4.2 to /system/pythonX.X.X. You can install both if you want. Installation size is about 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.

    After installation, the python director(ies) in /system will contain the static python binary, some scripts, and a bunch of modules and documents. Separate scripted executables will be installed to /system/bin/python or /system/bin/python3 depending on which one is installed. These basically just set the PYTHONHOME environment variable and execute the python binary.


    TESTING
    When you boot up to Android after installation, you should be able to just open up a terminal and fire off some python commands.

    Test using python 2.7.9:
    Code:
    python -c 'print "Hello World!"'


    Test using python 3.4.2:
    Code:
    python3 -c 'print("Hello World!")'


    You can also write scripts shelled with python:

    Code:
    #!/system/bin/python
    
    print "Hello World!"

    Make sure to set them as executable with "chmod +x".


    Adjust your screen brightness via /sys (root required, tested on Galaxy Nexus):
    Code:
    python -c 'f=open("/sys/devices/omapdss/display0/backlight/s6e8aa0/brightness","w"); f.write("40"); f.close()'

    A cool script I made to tweak file system I/O like rq_affinity, rotational, etc for I/O blocks:
    Code:
    #!/system/bin/python
    
    import os,re,sys
    
    list=[]
    # find all directories containing rq_affinity
    for roots, dirs, files in os.walk('/sys'):
      for file in files:
        match=re.search(r'\S+/rq_affinity',os.path.join(roots,file))
        if match:
          list.append(match.group().replace('rq_affinity',''))
    
    # write specific values to files in each directory found before
    for dir in list:
      for name in 'rq_affinity', 'rotational', 'read_ahead_kb', 'nr_requests', 'iostats', 'nomerges', 'add_random':
        try:
          f=open(dir+name,'w')
          if name is 'rq_affinity': f.write('1')
          elif name is 'read_ahead_kb': f.write('512')
          elif name is 'nr_requests': f.write('512')
          else: f.write('0')
          f.close()
        except IOError:
          sys.stderr.write('Problem writing to ' + dir+name + '\n')

    Or execute "python" or "python3" without any parameters to open up the interpreter and go from there. When you start for example "python" (python 2.7.8), you should see something like:

    Code:
    Python 2.7.8 (default, Dec  2 2014, 05:15:18) 
    [GCC 4.9.1] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>>


    ***I haven't fully tested this yet, so let me know how it goes. I know basic commands work, but there are still some complexities in the compilation that may need to be figured out.***


    (update) -- Install Extra Packages/Modules via "easy_install" -- (root required)
    1) In a shell, remount /system read-writable with:
    Code:
    mount -o remount,rw /system
    2) Change directory to /system/pythonX.X.X, i.e.,
    Code:
    cd /system/pythonX.X.X
    3) Execute easy_install followed by a package name, i.e.,
    Code:
    ./easy_install [i]package_name[/i]
    4) Let it download and install! :) Then test it out.
    5) Remount /system read-only with:
    Code:
    mount -o remount,ro /system

    Also, to remove a package, execute:
    Code:
    ./easy_install -m [i]package_name[/i]
    rm -r ../lib/pythonX.X/site-packages/[i]package_name[/i]*.egg

    *** Flashing the resolv.conf patch may be required to make an internet connection to download modules.

    Also, a few things need to be done to get easy_install working:
    1) For python 2.7.8 or 2.7.9 you need to link python2.7 to python like this:
    Code:
    ln -s /system/python2.7.9/bin/python2.7 /system/python2.7.9/bin/python
    I'll make sure this is automatically done in future installs.

    2) SSL certificates need to be added to "/etc/pki/tls/certs/" to use SSL with easy_install.
    Code:
    mkdir -p /etc/pki/tls/certs
    curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt


    ISSUES
    The python installations have the following modules compiled in with external dependencies such as libraries not included at this point:
    Code:
    --------------PYTHON 2.7.8-2.7.9----------------------
    _hashlib        _multiprocessing     _ssl             
    _testcapi       bz2                   crypt
    dbm
    Code:
    --------------PYTHON 3.4.2-----------------------------
     _crypt             _dbm                 _decimal           
    _hashlib            _multiprocessing     _ssl
    _testcapi
    I will look into this more.

    As @bubbleguuum points out, name resolution with python 3.4.2's urllib wasn't working. I found a workaround by adding the line
    Code:
    options single-request-reopen
    to /system/etc/resolv.conf. This still needs some testing but seemed to work for me. I've included a recovery flash zip to patch /system/etc/resolv.conf if the line is not there since this is a root operation.

    UPDATES
    * 2014-12-02 * Compiled in more modules for each installation. This makes the installation a bit bigger, but it's worth it. You get a more complete python! :) To get stuff like help functions and math and readline modules, you need the larger installations linked below under "DOWNLOADS".
    * 2014-12-03 * Thanks @cybojenix for pointing out some incapatibility issues with the original edify installer I was using. I have now updated the installers to use SuperSU's great non-edify sh-script installer.
    * 2014-12-05 * Some updates to the installers to include more modules, plus size reduction for python 3.4.2. Modules added to python 2.7.8 : _bsddb _ctypes, _ctypes_test, _hotshot, _json, _lsprof, _sqlite3, future_builtins, and ossaudiodev, easy_install, pip(broken for now). Modules added to python 3.4.2: _bz2, _ctypes, _ctypes_test, _gdbm, _json, _lsprof, _opcode, _sqlite3, _testbuffer, _testimportmultiple, ossaudiodev, xxlimited
    * 2014-12-05 * Some big updates to the installer to include almost the rest of the modules that wouldn't compile. Also easy_install is working on my end with these new experimental builds. With a little effort, should be able to get pip working as well. I had to use some hacky flags to get it to compile, ignore unresolved-symbols and such, so it definitely needs some testing. Another thing I did was tweak the install script to write over previous installations from these installations. I recommend downloading the experimental builds if you read this. The worst thing that might happen is one of the new modules I've included won't work completely.
    * 2014-12-14 * Added python 2.7.9. Trimmed the installation sizes down to the much more reasonable 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
    * 2014-12-24 * Attached a recovery flash.zip to patch resolv.conf to try and fix a urllib name resolution error.
    * 2014-12-26 * A couple updates: Tweaked python installations so that modules looks for "/system/bin/sh" instead of "/bin/sh" when needed. Also a few "#!" corrections for scripts in the python bin directory. This shouldn't break anything from what I've tested, only make Android python more capable, but I'll keep the old installations attached for now just in case since the modifications involved using "sed". The second update is nice. I've added a little section on installing extra modules with easy_install(included and working with all installations).
    5
    Compiling
    The environment I compiled in was a debian "wheezy" image mounted as a loop device on Android. The reason I compile this way is so I have full access to the arm environment tools, gcc compiler, etc. without having to go through the trouble of using a cross-compiler (which doesn't work in this case from my testing with uClibc, or eglibc. Code sourcery's might work but I didn't feel like booting an x86 linux installation to try). You can recreate the same environment pretty easily by using either Complete Linux Installer or Linux Deploy apps. I use both.

    To get a static python compiled, I first downloaded the following packages with APT:
    Code:
    [i]apt-get build-dep python python3[/i]
    build-essential gcc make # building tools
    binutils-gold # awesome gold linker
    zlib1g-dev # zlib
    libreadline-dev, libncurses5-dev # readline, curses
    libbz2-dev # bz2
    libsqlite3-dev # sqlite3
    python-bsddb3, python3-bsddb3
    libgdbm-dev # gdb
    libssl-dev # ssl
    python-tk, python3-tk # tkinter
    libdb-dev # db
    python-gdbm python-bsddb3
    libffi-dev # _ctypes
    tcl8.6-dev # tkinter
    libx11-dev # tkinter
    libmpdec-dev # decimal

    Then I manually installed binutils-gold by unpacking the rpm for armv7h (this isn't available with APT yet). You can get it here though. (Update: this should be available in the debian repositories now.)


    For static compilation, you need to make some changes to Modules/Setup from the python source directory. Specifically, you need to add
    Code:
    *static*
    to the top of the file. Then you need to uncomment any commented modules you would like compiled in. When you compile and see a list of failed modules, search for each of these in Modules/Setup and uncomment them. For example, change
    Code:
    #math mathmodule.c _math.c
    to
    Code:
    math mathmodule.c _math.c
    . Then recompile. There may be some shared dependencies and packages as well you need to figure out to get them to compile. Take a look at the attached "Setup" files for more detail.



    The actually compile steps look like this:
    (1)
    Code:
    ./configure --build=arm --prefix="$PWD"/out LDFLAGS="-static -static-libgcc -Wl,--unresolved-symbols=ignore-all -Wl,--export-dynamic" CPPFLAGS=-static CXXFLAGS=-static CFLAGS="-Os -static" LDFLAGS=-static LD=ld.gold
    (2) Modify the Modules/Setup file.
    (3)
    Code:
    make clean; make install



    After compiling, there are some things you can do to make your installation in "./out" smaller. Strip the large binaries in "out/bin", i.e.
    Code:
    strip -s ./out/bin/python2.7
    Also remove the python archive library, *.o's, etc. with:
    Code:
    make clean
    This might seem kind of weird, but it works and cleans out unnecessary files from your installation.



    **Make sure to backup your Modules/Setup file if you run "make distclean". I lost my 3.4.2 Modules/Setup file after doing this. :silly: Now I gotta recreate it, dammit.
    **I should also mention that compile time is very fast given the amount of data generated. It takes about 10 minutes to get python and all it's modules compiled on my Galaxy Nexus.**

    :EDITS:
    : Updated some configure parameters. Removed unnecessary, extra "./configure". Also added some updates to the included "Setup" files.
    : Hacky update to "./conifgure" to ignore unresolved symbols, export dynamic. This allows certain modules to be compiled which have shared dependencies.
    : Update to trimming down the installation size after "make". Use another "make clean".

    Downloads - Extract in the source directory
    Setup (python 2.7.8)
    Setup (python 3.4.2)
    Setup (Experimental) (python 2.7.8)
    Setup (Experimental) (python 3.4.2)
    1
    Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.

    Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.

    Give me a shout if you need anything. I hope to see even more come out of this thread ;)

    Edit:
    I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
    1
    @7175

    Thank you very much for these binaries.

    There is however a problem: name resolution (DNS) doesn't seem to work at all (with both python 2 and 3 downloads).
    All attempts to use urllib.request.urlopen('http://somehost.com') fail with "<urlopen error [Errno -2] Name or service not known>" (running python as root but it doesn't matter, and a rooted Nexus 4 running 4.4.4 stock ROM):

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
    File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 455, in open
    response = self._open(req, data)
    File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 473, in _open
    '_open', req)
    File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
    File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1202, in http_open
    return self.do_open(http.client.HTTPConnection, req)
    File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1176, in do_open
    raise URLError(err)
    urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

    it works if replacing somehost.com by its ip address.

    To reproduce above stack trace with python 3:

    import urllib.request
    urllib.request.urlopen('http://www.google.com')

    while this work:

    urllib.request.urlopen('http://173.194.45.229')

    nslookup and general name resolution work anywhere else:

    root@mako:/ # nslookup www.google.com
    nslookup wwwgoogle.com
    Server: 8.8.4.4
    Address 1: 8.8.4.4 google-public-dns-b.google.com

    Name: wwwgoogle.com
    Address 1: 2a00:1450:4007:80c::1014 par03s15-in-x14.1e100.net
    Address 2: 173.194.45.242 par03s15-in-f18.1e100.net
    Address 3: 173.194.45.240 par03s15-in-f16.1e100.net
    Address 4: 173.194.45.244 par03s15-in-f20.1e100.net
    Address 5: 173.194.45.241 par03s15-in-f17.1e100.net
    Address 6: 173.194.45.243 par03s15-in-f19.1e100.net

    This issue has probably something to do with how python was compiled to do name resolution. I remember vaguely a similar issue
    on Ubuntu 12.04, with a ffmpeg compile that would always fail to resolve hostnames in URLs. No sure what the exact fix was but it had something to do with the libc and its name resolution mechanism. Or maybe something ipv6 related ?
    1
    @bubbleguuum : Thanks for checking out that important issue.

    I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
    Code:
    options single-request-reopen
    to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.

    Thank you for looking into this so fast.

    My intended usage is for running the python binary from a regular non-root app (which works great!), so editing resolv.conf is not an option.