5,599,478 Members 39,391 Now Online
XDA Developers Android and Mobile Development Forum

Bake GNU/Linux into Android [Work-In-Progress]

Tip us?
 
Edge-Case
Old
(Last edited by Edge-Case; 12th January 2014 at 04:28 AM.)
#1  
Edge-Case's Avatar
Member - OP
Thanks Meter 25
Posts: 97
Join Date: Jul 2013
Angry Bake GNU/Linux into Android [Work-In-Progress]

How to add glibc stack to Android, bringing out the Penguin within.


Important note:
I am still working out the kinks, learning, etc. This is not ment to be a daily use rom, esspecially what I put up for downloading and testing. Consider them teasers to get you to build your own.


Downloads:
This is the base/root directory of my shared google drive.

This is the 1st build that postgres worked -Note that bluetooth did not work, thats all I noticed though. Also, the scripts have not been debugged that are in this zip, replace them with the ones attached to this post, they are the result of debugging this zip.

This is the Kali rootfs using Debian repo to fix some missing packages. You can either use this root-fs, make your own, or download one, I added a link to a debian-wheezy fs below in the Notes.
EDIT: Is working now, remember to replace the scripts with the debuged ones or edit them to suit your prefrences.

Blank compressed .img file, gunzip it. -Note: It is a 4 gig img file.

And a special I am sorry for anyone who downloaded anything in the last hour or so. -E.C. 1.11.14
This is what is looks like when you get it set-up correctly, if you encounter errors, please forgive me.

If at the first time, it seems to not have worked, wait another minute. Depending on how big your root-fs is it will take some time.
If you replaced the scripts and placed everything in the correct dirs, write a reply to let me know.
Use "getprop persist.cadkl.cfboot" to check the process of the first boot. If it is one then either it is in the process of running the firstboot.sh script or it failed before finishing that script, if it is at 2 , then it passed that and went to submount.sh, just be patient the first time around. I recently thought it wasn't working and started messing around only to find shortly later that things were being mounted as I was deleting them. I need to work out a better mechanism, perhaps one with a process bar and an error log.
In order to use this ROM you must have a directory on your sdcard called "cadkl", this is "/storage/emulated/legacy/cadkl" or "/sdcard/cadkl" or "/storage/sdcard0/cadkl" and within cadkl dir must be "cadkl-fs.tar.gz" and an image called "cadkl.img". It can be blank at first or full of junk, what is on the image will get erased the first time you "setprop persist.cadkl.autofs 1" from the command-line. That setprop command will set everything up for you. If it doesn't, let me know. In the future I'll mess with Androids settings menu to put this all in a gui for just one button set-up like allowing adb-debugging. I do suggest you atleast read some of the notes below, even if you don't want to buid your own version.


Notes:


-Added tarball with Android.mk and cadkl-autofs to make things simple, just extract and put it in your source tree.
-Note that you still need to modify the filesystem as described below to tell Android about all the new branches and some important leaves.

-mount scripts at bottom with init.cadkl.rc
-init.cadkl.rc goes in / - You can make an Android.mk file to put it there or by unpacking initramfs.cpio.gz and editing there, dont forget to import -from init.rc, I do it right after cyanogenmod does, find it in ${a_src_rt}/system/core/rootdir/init.rc
-I put cadklinitfs.sh in /system/xbin/
-I put firstboot.sh and submount.sh in /system/etc/cadkl/lib/
-The mounting either requires the image file to be /sdcard/cadkl/cadkl.img or to edit the scripts
-The mounting either requires a /sdcard/cadkl/cadkl-fs.tar.gz lcontaining the glibc stack or to edit the scripts
-To start the just "setprop persist.cadkl.autofs 1" or what ever name you choose to change it to. Anything but 1 should stop it from mounting at boot.
-It takes a minute the first time to untar the archive and rsync everything into place.
-You may want to look into how this script treats /etc
- I use this Debian root-fs, either edit the scripts or unxz and re-gzip it and rename it to cadkl-fs.tar.gz
http://s3.armhf.com/debian/wheezy/de...0131129.tar.xz
- Or, I made a custom one as per the Kali docs . Hint: If a package fails to be located, I have luck using a debian repo with the kali repo, in fact thats how the name CyAnDeKaLi came about, CyanogenMod, Android, Debian, Kali, and Linux. A beefy chemistry to say the least.
- You'll also need at least a 2 gigabyte image file, it will get erased at first boot and replaced with the contents of the archive, be aware, you could start with a fresh one or an old one.

feel free to change and do what you like to whatever, if its called public domain then thats what it is, basically I don't give a ***** what you do. Also, I am not responible for what you do.

Build your own:


Blue = is to do / our additions
Green = stays the same / was already present
Red = is to remove / comment out
Orange = experimenting

Added these to
${a_src_rt}/build/core/legacy_prebuilts.mk
Code:
....
....
	bin \
	home \
	lib \
	media \
	opt \
	root \
	run \
	srv \
	tmp \
	usr \
	var \
add these to ${a_src_rt}/system/core/rootdir/Android.mk
Under this part:
Code:
...
...
DIRS := $(addprefix $(TARGET_ROOT_OUT)/, \
		sbin \
		dev \
		proc \
		sys \
	        system \
		data \
		bin \
		home \
		lib \
		media \
		opt \
		root \
		run \
		srv \
		tmp \
		usr \
		var \
        ) \
        $(TARGET_OUT_DATA)
I add the the directories to
"${a_src_rt}/system/core/include/private/android_filesystem_config.h"
under and between the following...:
Code:
...
...
/* Rules for directories.
** These rules are applied based on "first match", so they
** should start with the most specific path and work their
** way up to the root.
*/

##static struct fs_path_config android_dirs[] = {
   { 00770, AID_SYSTEM, AID_CACHE,  "cache" },
.....
.....
    { 00755, AID_ROOT,   AID_ROOT,   "bin" },
    { 00755, AID_ROOT,   AID_ROOT,   "home" },
    { 00755, AID_ROOT,   AID_ROOT,   "lib" },
    { 00755, AID_ROOT,   AID_ROOT,   "media" },
    { 00755, AID_ROOT,   AID_ROOT,   "opt" },
    { 00750, AID_ROOT,   AID_ROOT,   "root" },
    { 00755, AID_ROOT,   AID_ROOT,   "run" },
    { 00755, AID_ROOT,   AID_ROOT,   "srv" },
    { 00755, AID_ROOT,   AID_ROOT,   "tmp" },
    { 00755, AID_ROOT,   AID_ROOT,   "usr" },
    { 00755, AID_ROOT,   AID_ROOT,   "var" },
    { 00755, LID_LIST,   LID_LIST,   "var/list" },// I wonder if sticky bits should be set on these dirs?
    { 00755, AID_ROOT,   LID_MAIL,   "var/mail" },// I am testing 01770, AID_ROOT, LID_$USER setttings so that only root n user can access
    { 00755, LID_MAN,    AID_ROOT,    "var/cache/man" },
    { 00755, LID_LP,     LID_LP,     "var/spool/lpd" },
    { 00755, LID_NEWS,   LID_NEWS,   "var/spool/news" },
    { 00755, LID_UUCP,   LID_UUCP,   "var/spool/uucp" },
    { 00755, LID_WWW_DATA,   LID_WWW_DATA,   "/var/www" },
    { 00755, LID_BACKUP,   LID_BACKUP,   "var/backups" },
    { 00755, LID_IRC,    LID_IRC,    "var/run/ircd" },
    { 00755, LID_GNATS,  LID_GNATS,  "var/lib/gnats" },
    { 00755, LID_LIBUUID,  LID_LIBUUID,   "var/lib/libuuid" },
    { 00755, LID_MESSAGEBUS,  LID_MESSAGEBUS,   "var/run/dbus" },
    { 00755, LID_COLORD,  LID_COLORD,  "var/lib/colord" },
    { 00755, LID_SSHD,   LID_SSHD,   "var/run/sshd" },
    { 00755, LID_STATD,  LID_STATD,  "var/lib/nfs" },
    { 00755, LID_POSTGRES, LID_POSTGRES,  "var/lib/postgresql" },
    { 00755, LID_MIREDO,   LID_MIREDO,    "var/run/miredo" },
    { 00755, LID_STUNNEL4, LID_STUNNEL4,  "var/run/stunnel4" },
    { 00755, LID_IODINE,   LID_IODINE,    "var/run/iodine" },//end of sticky bit question?
    { 00644, AID_ROOT,   AID_ROOT,   0 },
};
Feel free to modify the permissions as you see fit, I am no professional. I was trying to be permissive at first and then tight up after things started working.

Next is also in ${a_src_rt}/system/core/incude/private/android_filesystem_config.h
Code:
/* Rules for files.
** These rules are applied based on "first match", so they
** should start with the most specific path and work their
** way up to the root. Prefixes ending in * denotes wildcard
** and will allow partial matches.
*/
static struct fs_path_config android_files[] = {
    { 00440, AID_ROOT,      AID_SHELL,     "system/etc/init.goldfish.rc" },
    { 00550, AID_ROOT,      AID_SHELL,     "system/etc/init.goldfish.sh" },
    { 00440, AID_ROOT,      AID_SHELL,     "system/etc/init.trout.rc" },
    { 00550, AID_ROOT,      AID_SHELL,     "system/etc/init.ril" },
.....
.....
.....
    { 00755, AID_ROOT,      AID_ROOT,      "lib/*" },
    { 00755, AID_ROOT,      AID_ROOT,      "usr/bin/*" },
    { 00755, AID_ROOT,      AID_ROOT,      "usr/local/bin/*" },
    { 00750, AID_ROOT,      AID_ROOT,      "usr/local/sbin/*" },
    { 00750, AID_ROOT,      AID_ROOT,      "usr/sbin/*" },
    { 00664, LID_LIST,      LID_LIST,      "var/list/*" },
    { 00664, AID_ROOT,      LID_MAIL,      "var/mail/*" },
    { 00664, LID_MAN,       AID_ROOT,       "var/cache/man/*" },
    { 00664, LID_LP,        LID_LP,        "var/spool/lpd/*" },
    { 00664, LID_NEWS,      LID_NEWS,      "var/spool/news/*" },
    { 00664, LID_UUCP,      LID_UUCP,      "var/spool/uucp/*" },
    { 00664, LID_WWW_DATA,   LID_WWW_DATA,   "/var/www/*" },
    { 00664, LID_BACKUP,    LID_BACKUP,    "var/backups/*" },
    { 00664, LID_IRC,       LID_IRC,       "var/run/ircd/*" },
    { 00664, LID_GNATS,     LID_GNATS,     "var/lib/gnats/*" },
    { 00664, LID_LIBUUID,   LID_LIBUUID,   "var/lib/libuuid/*" },
    { 00664, LID_MESSAGEBUS,  LID_MESSAGEBUS,   "var/run/dbus/*" },
    { 00664, LID_COLORD,    LID_COLORD,    "var/lib/colord/*" },
    { 00664, LID_SSHD,      LID_SSHD,      "var/run/sshd/*" },
    { 00664, LID_STATD,     LID_STATD,     "var/lib/nfs/*" },
    { 00664, LID_POSTGRES,  LID_POSTGRES,  "var/lib/postgresql/*" },
    { 00664, LID_MIREDO,    LID_MIREDO,    "var/run/miredo/*" },
    { 00664, LID_STUNNEL4,  LID_STUNNEL4,  "var/run/stunnel4/*" },
    { 00664, LID_IODINE,    LID_IODINE,    "var/run/iodine/*" },
    { 00664, AID_ROOT,      AID_ROOT,      "var/*" },
    { 00644, AID_ROOT,      AID_ROOT,       0 },
};
Make sure to define them and add them to the array! The new users and daemons that is...
Code:
/* This is the master Users and Groups config for the platform.
** DO NOT EVER RENUMBER.
*/

#define AID_ROOT             0  /* traditional unix root user */

#define LID_DAEMON           1  /* Linux Damemon user */
#define LID_BIN              2  /* Linux bin user */
#define LID_SYS              3  /* Linux sys user */
#define LID_SYNC             4  /* Linux sync user */
#define LID_GAMES            5  /* Linux games user */
#define LID_MAN              6  /* Linux man user */
#define LID_LP               7  /* Linux mail user */
#define LID_MAIL             8  /* Linux mail user */
#define LID_NEWS             9  /* Linux News user */
#define LID_UUCP            10  /* Linux uucp user */
#define LID_PROXY           13  /* Linux proxy user */
#define LID_WWW_DATA        33  /* Linux www-data user */
#define LID_BACKUP          34  /* Linux backup user */
#define LID_LIST            38  /* Linux Mailing List Manager */
#define LID_IRC             39  /* Linux irc user */
#define LID_GNATS           41  /* Linux Gnats Bug-Reporting System (admin) user */

#define LID_LIBUUID        100  /* */
#define LID_MESSAGEBUS     101  /* */
#define LID_COLORD         102  /* colord management daemon */
#define LID_SANED          103  /* */
#define LID_SSHD           104  /* sshd daemon */
#define LID_STATD          105  /* */
#define LID_POSTGRES       106  /* PostgreSQL administrator */
#define LID_MIREDO         107  /* */
#define LID_STUNNEL4       108  /* */
#define LID_IODINE         109  /* */
#define LID_SSLH           110  /* */

#define AID_SYSTEM        1000  /* system server */

#define AID_RADIO         1001  /* telephony subsystem, RIL */
#define AID_BLUETOOTH     1002  /* bluetooth subsystem */
....
....
....

static const struct android_id_info android_ids[] = {
    { "root",      AID_ROOT, },
    { "system",    AID_SYSTEM, },
    { "radio",     AID_RADIO, },
....
....
    { "daemon",    LID_DAEMON, },
    { "bin",    LID_BIN, },
    { "sys",    LID_SYS, },
    { "sync",    LID_SYNC, },
    { "games",    LID_GAMES, },
    { "man",    LID_MAN, },
    { "lp",    LID_LP, },
    { "mail",    LID_MAIL, },
    { "news",    LID_NEWS, },
    { "uucp",    LID_UUCP, },
    { "proxy",    LID_PROXY, },
    { "www-data",    LID_WWW_DATA, },
    { "backup",    LID_BACKUP, },
    { "list",    LID_LIST, },
    { "irc",    LID_IRC, },
    { "gnats",    LID_GNATS, },
    { "libuuid",    LID_LIBUUID, },
    { "messagebus",    LID_MESSAGEBUS, },
    { "colord",    LID_COLORD, },
    { "saned",    LID_SANED, },
    { "sshd",    LID_SSHD, },
    { "statd",    LID_STATD, },
    { "postgres",    LID_POSTGRES, },
    { "miredo",    LID_MIREDO, },
    { "stunnel4",    LID_STUNNEL4, },
    { "iodine",    LID_IODINE, },
    { "sslh",    LID_SSLH, },
};
*This is what uid's my /etc/passwd had, some distro's may be different then others, when in doubt check your /etc/passwd . If I understand the source in this android_filesystem_config.h file, the uid's and gid's are the same, so one might need to script usermod to update the /etc/passwd file that the linux programs parse to determine uid and gid values to reflect what gets hardcoded into the Android filesystem during a builid, for an Admin turning your phone into a server of some sorts, it will require much of the configuration pre-build for the Android side of things as things stand now, this is the reason for the changes we are making here, "hard-coded pre-configuration". Ideally /etc/passwd /etc/groups and other relevant files will mirror each other, not in syntax, but the values should be the same. This is until a mechanism to update the filesystem's uid/gid database with GNU's/Linux's new users as they get added. Also, on the GNU/Linux side, user addition must be modified to follow the offset specified by Android because the droid uses 1000's range for system users, and I think thats what range is used for new "users" in GNU/Linux. Once all these things are figured out it should be more friendly to power users and perhaps even servers. I am not sure about the whole "sandbox" scenario, it seems to go against what Unix was after, small modular programs that can work together to serve a larger function. Android, on the other hand, seems to isolate programs from each other like a prison. So perhaps a balance can be found, or we can unlock the doors but not tell the inmates.

And add some properties to test for first boot so you can have an init service (call a shell script from init) mount the files inplace but make sure to wait for /dev/loop0 and a few other things.

I'll dig through the source again and find my changes to the kernel and some other things, all I did to the kernel was comment out the parts that happen if ANDROID_PARANOID_NETWORK is defined, since I seem to break my builds when I configure the kernel I just did it by hand so to speak. Just cgrep the from the kernel source tree for ANDROID_PARANOID_NETWORK. (-Fri Jan 10, 3:40pm : I just noticed my bluetooth isn't working, wifi is, cellular network shows bars but I haven't had service in months so I am not sure if that is actually working. When I attempt to turn on my buetooth, it turns off right away, I am not sure if I caused this or if it was an existing bug, either way its an issue, may be permission related...)

This is where I leave off for tonight : Thu Jan 9, 11:18 PM

Things to fix:

-User mechanisms need to work together or become one.

-Some environmet settings seem to affect which "side" works right. Sometimes nano needs config on gnu side, sometimes depending on which PATH dirs are searched first some programs break. Basically some of Android's programs expect output from other programs to be parsed a certain way, this can be fixed.

-Log-in's! I would like to eventually get rid of Android's su infavor of GNU's su which requires password authentication.

-I would like the user's Log-in password work to unlock the gui like gnome. But maybe a soft lock and hard lock, where a soft lock only requires a swipe to unlock the gui and one could hard-lock it by swiping to an option on the soft-lock screen, we should all know what screen I am refering to, we don't use the shell that much.

-Perhaps Su can be unlocked and a password set from a settings menu, similar to the developers options on CM.

-I would like to see multi-user support similar to the GNU/Linux set-up but like KDE, root should never log-in to the gui or launcher (Tebuche ?), but only be accessed via the shell.

Any suggestions or improvements are very welcome.

--------------original post---------------
I am having a bit of a hard time configuring the kernel the "Android Way" with menus or defconfigs so I just greped through the source and changed what I needed to there, messy, but hey, it worked.

Basically, for now I am comfortable with it but I will need to find a "better way".

I got postgresql working! YAY!!!!

It all started when I noticed that the metasploit console worked in Android's root when /lib and everything else is in place. The only problem was that I couldn't get a socket to open for the life of me. I disabled Paranoid Networking in the source. I hope to re-enable it once I get it working with Linux's groups and programs that need sockets, but that will take awhile.

So now that postgresql can open a socket and bind to a port and all that, msfconsole will be able to connect to a database!!!! WHOOHOOO

This will eventually turn into a how-to build a rom with a glibc stack. My ultimate goal is to get bionic compatible with glibc and to include c++ exception handleing thus eleminating the need to have two libs, or I can make glibc compatible with bionic, thats in the air.

Time, to do some field testing, if anybody wants the rom I just made let me know, it also requires a tar.gz file containing the gnu/linux fs. The tar.gz gets unzipped into a cleaned .img file via an init service that calls a script, so its all configurable by root. It won't mount the first boot, it gives you a chance to push and pull things around, then just "setprop persist.cadkl.autofs 1" and wait some seconds, after about 20-30 second /bin, /usr/ etc etc become populated and that will happen automagically during boot after the first if the prop is set to 1.

Let me know,
Peace out,
Edge-Case

Update: Jan 9 12:40
I built ruby and metaslpoit last night ( compiled on my phone )
Here is a pic of me finally connecting to the database on my phone! http://forum.xda-developers.com/atta...1&d=1389292952
Attached Thumbnails
Click image for larger version

Name:	Screenshot from 2014-01-09 02:30:01.png
Views:	2898
Size:	108.0 KB
ID:	2504015   Click image for larger version

Name:	success.png
Views:	114
Size:	96.0 KB
ID:	2504878   Click image for larger version

Name:	neo_droid.jpg
Views:	36
Size:	165.8 KB
ID:	2510040   Click image for larger version

Name:	Screenshot from 2014-01-11 21:26:41.png
Views:	110
Size:	188.0 KB
ID:	2510059   Click image for larger version

Name:	Screenshot from 2014-01-11 22:26:38.png
Views:	106
Size:	118.5 KB
ID:	2510185  

Attached Files
File Type: txt init.cadkl.rc.txt - [Click for QR Code] (438 Bytes, 2 views)
File Type: txt cadklinitfs.sh.txt - [Click for QR Code] (1.4 KB, 2 views)
File Type: txt firstboot.sh.txt - [Click for QR Code] (1,015 Bytes, 1 views)
File Type: txt submount.sh.txt - [Click for QR Code] (675 Bytes, 1 views)
File Type: gz cadkl-autofs.tar.gz - [Click for QR Code] (1.7 KB, 0 views)
"Wise man say, 'Forgiveness is divine,
but never pay full price for late pizza'."

Some Usefull links:

http://elinux.org/Android_Kernel_Features // Good read on the Android specificities of the kernel

http://www.androidenea.com/2010/03/s...binder-in.html // Info on Binder and ashmem and how to share memory!
The Following 2 Users Say Thank You to Edge-Case For This Useful Post: [ Click to Expand ]
 
1BadWolf
Old
#2  
1BadWolf's Avatar
Senior Member
Thanks Meter 445
Posts: 816
Join Date: Aug 2012
Location: Nowhere, Louisiana
Sounds interesting. Count me in for testing. :thumbup:

If I had a dollar for every time I said that, I'd be making money in a very weird way.
The Following User Says Thank You to 1BadWolf For This Useful Post: [ Click to Expand ]
 
Edge-Case
Old
#3  
Edge-Case's Avatar
Member - OP
Thanks Meter 25
Posts: 97
Join Date: Jul 2013
Default bump

Quote:
Originally Posted by 1BadWolf View Post
Sounds interesting. Count me in for testing. :thumbup:

If I had a dollar for every time I said that, I'd be making money in a very weird way.
Where can I upload some files to for free?
Apperently my root-fs's and zips are too big...
"Wise man say, 'Forgiveness is divine,
but never pay full price for late pizza'."

Some Usefull links:

http://elinux.org/Android_Kernel_Features // Good read on the Android specificities of the kernel

http://www.androidenea.com/2010/03/s...binder-in.html // Info on Binder and ashmem and how to share memory!
 
CNexus
Old
#4  
CNexus's Avatar
Recognized Contributor
Thanks Meter 11065
Posts: 8,411
Join Date: May 2012
Location: /home/cnexus/android

 
DONATE TO ME
Quote:
Originally Posted by Edge-Case View Post
Where can I upload some files to for free?
Apperently my root-fs's and zips are too big...
Google drive
Sign up for the new Copy file storage service with this link and get both of us an extra 5 GB!
Or sign up for Dropbox with this link and we can both grab some extra storage!

The Following 2 Users Say Thank You to CNexus For This Useful Post: [ Click to Expand ]
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes