[Shell][mksh][Scripting]::mksh-R50e-static-FULL+mkshrc_mods:::

:::BRIEF DESCRIPTION:::
mksh is the official, currently-developed shell used for Android. It is not an app nor a virus (lol) like some people seem to think, looking at a quick xda search. And as the core shell for android, it is used to shell superuser, so if you're rooted and get a message saying "mksh requests root", that basically just means that superuser is being run correctly and is using the default Android shell that is in every Android device. (GOOD)
If you get a message saying bash(or any other shell) requested root, and you're NOT in the terminal interactively. That most likely is NOT GOOD! Almost all bash scripts I've seen will run shelled with mksh, so be safe and shebang with mksh. You'll have better performance and features, and when you see bash requesting root, you'll know when your security is likely compromised.


::: mksh Resources/links/info :::

mksh author: mirabilos (on xda forums)

mirbsd mksh main page: https://www.mirbsd.org/mksh.htm

mksh man page: https://www.mirbsd.org/htman/i386/man1/mksh.htm

edited 15Aug2014
Now that that's out of the way, I've attached a few zips in the 2nd post. The first contains my personalized mkshrc (startup mksh script), the full original mkshrc it's based on, and a diff for anyone who's interested. **This file belongs at /system/etc/mkshrc.** The rest of this first post details what I've cooked up in my mkshrc (and fpath functions). The other zips' contents are described in the 2nd post.


Features scripted into mkshrc:

--sudo: basic su executor
e.g.:

--suid: (set user id) Uses chpst to run a prog setsuid, this is broken in most busyboxes so I included one with a working chpst. Usage:::
e.g.:

--Rebind tab completion: unjams clustered words:
e.g.: <[cursor]> is cursor position

--Rebinded "control + u" to "cut-to-beginning of line" like bash. (As mirabilos points out, this is ksh incompatible behaviour since "control + u" is "kill-line" in *ksh shells. This can be removed from the (/system)/etc/mkshrc shell startup file if it presents any problems in editing.) The exact binds are:

--TMPDIR: mounts a tmpdir at /tmp for all users and random generates subdirs for individual secured tmpdirs. This fixes here-docs, here-strings, and here-executions.
e.g.:

--Basic "help" command: Android lacks this and the doc is still under development I believe. I added all the basic info/command-forms for mksh bultins to a command help(aliased to h):
e.g.:
or
(all help is printed, grep from here if you want)

--Complicated-programmed and colorful PS1 prompt, it looks good and I thought it through. Programming shows mksh potential for rich shell programming. It auto-turns off xtrace within itself (to 3 lines) so that 50 lines don't get printed each return with set -x. mksh small (R43 mksh) will be autodetected and give a different white and black classic ksh prompt.

--ton of ultra lazy aliases, got my android scripting speed up to ~70% my normal pc speed.

--dot.mkshrc functions from official mkshrc like pushd, popd, dirs, more, hd, Lb64encode, Lb64decode, hash optimizations

The scripting in this gets pretty complex, especially the inherent mkshrc functions. It would make a good scripting learning tool to say the least. I know I've learned twice what I knew from bash scripting back 3 months ago.


I've been working on modding the mkshrc a while, so I appreciate any error reports if you give mine a try. Shouldn't be any problems though.

Modifying the mkshrc file / mksh shell has system-wide effects, so PLEASE nandroid backup as usual!!! This should be COMPATIBLE with all androids, but has only been tested on my Galaxy Nexus thus far. **The binary is compatible with androids using an ARM cpu, but the steps in the 3rd post will generate a binary for whatever ARCH you're running.**



EDIT: reprogrammed sudo with pexec, so it can do parallel executions. pexec, grep, and a full busybox are included in a zip with some other bins.

For Jack Palevich's terminal emulator or connectbot, telnet, etc, I use this as my su'd "shell" command under >>Options>>Shell:

What's safer though is to just use a non-su shell command:
...and then use sudo for your su commands. This is more linux'y.


EDIT2 (big update): Got static mksh R50 compiled. Ended up booting ubuntu in Android to compile instead of cross-compiling. mksh's Build.sh compiler script works great. I just had to "sudo apt-get install gcc" and bam it built with "LDSTATIC=-static ./Build.sh".
EDIT3 - Changed main.c source mkshrc file to /system/etc/mkshrc, perfect static mksh R50 now.
EDIT4 - Changed compiler to klcc to get size way down, now at 196KB with -O2 LTO speed optimizations.
EDIT5 - Binary is updated to R50d, compiled with CFLAGS=-Os, and stripped to get size down to ~140KB. (This is even smaller than the R43 dynamic mksh binary that is currently used in roms.)
EDIT6 - Added a mksh trunk (latest and greatest) binary with the printf builtin compiled in. Use with caution. Btw, if you read all the way down here, then yes, these binaries work in lollipop since they're statically compiled.

:ownloads:: (#7 or #8 is the static binary by itself)

(1) mksh R50 source, binary, manual

(2) Highly compressed pack of extra bins/libs that are mentioned in my mkshrc (zsh, lz4c, grep, busybox, bash, nano, vim, curl, etc) and some other cool ones like macchanger and ed. Download #3 to extract on Android.

(3) 7za in case you need it to extract (2) on Android.

(4) Classic unix text editor "ed" which mksh uses for edit functionality. (Copy to /system/bin along with mksh.)

(5) Official mkshrc (unzip and copy to /system/etc, set permissions to 0444)

(6)

• My personalized mkshrc startup mksh shell script based on the official full mkshrc 2014/01/11 release. Described in first post. If you try this, make sure to symlink /system/xbin/busybox to /system/bin/busybox.ndk.
• Official mkshrc 2014/01/11 release
• "diff -ur /system/etc/dot.mkshrc--official.mkshrc mkshrc" (diff patch set from official mkshrc to my personalized mkshrc)

(7) mksh static binary! (Just unzip, "chmod +rx" it, and copy to /system/bin or run from anywhere, no libs required )

(8) mksh R50e with printf builtin (I know some people were looking for this)

::: Building mksh Static On Android (really Ubuntu though..):::

This is just for anyone who wants to try, and especially those without access to a PC.

Things we'll need besides your phone:
-"Android Terminal Emulator" app {get it from (http://jackpal.github.com/Android-Terminal-Emulator/downloads/Term.apk) or fdroid or gplay}
-"Complete Linux Installer" app {get it from (http://sourceforge.net/projects/linuxonandroid/?source=typ_redirect) or gplay or maybe fdroid}
-internet/wifi
-mksh source {download from (https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R50.tgz) or link in first post or DL in second post}


1) We need to get Ubuntu or Debian booted for a sufficient build environment. I've used both on Android but I like the better stocked terminal in the Ubuntu images. I used the app Complete Linux Installer which is free and works beautifully, very simple and easy too. In the app you want to follow the short instructions to download an Ubuntu image, rename it to ubuntu.img, and place it in a folder named ubuntu in /sdcard. Then hit menu in the app and click the terminal screen icon that says "Launch". An Ubuntu terminal will now open in Android Terminal Emulator. Super quick and easy.

2) Let's download GCC and ed so we can build and test with zero errors.

3) Now the cool thing about this chroot Ubuntu environment is that we still have access to the sdcard to transfer files between Android environment and the chroot jail. Extract your downloaded mksh source to your Ubuntu home with something like:

4) Since we're building this for Android, the mkshrc file is at /system/etc/mkshrc instead of ~/.mkshrc, so we can change this with:

Now we can build mksh statically:

Let the build script crank out the binary for a couple minutes and run the test suite. There should only be an error on a "mkshrc" test since we changed it for android. The "-r -c lto" flags/opt aren't needed but recommended.

Edit: (size optimization): Strip the compiled mksh binary to make it super small (~150KB with a CC="klcc" or CC="diet gcc" compile).

5) Now let's copy it to /system/bin to replace your old one.

.. and done. Run some scripts and enjoy your static mksh!



*** OPTIONAL EXTRA STEPS TO USE klcc or other compiler ***
In step (1), also do
to get the klibc development tools, and then...
In step (4), this is how you specify to use klcc (aka build against klibc using the klcc wrapper):


:::Compile mksh in Android (for ARM):::
1) Download the attached cross-compiler and mksh source (https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R50.tgz). Copy them to /data/media. If you don't have 7za, download that as well from the previous post but place it in /system/bin and "chmod +x" it

2) Extract contents:

3) Compile the source with:

4) Replace old mksh and mkshrc

Hey E.VA. thanks for checking out the thread. If you want to just test out R48 mksh, just copy(from zip root directory) the lib*.so from /data/media/shell to /system/lib and then copy /data/media/shell/mksh to /system/bin (and chmod 0555 /system/bin/mksh; chmod 0444 /system/lib/*.so). Then you'll have your whole system running full mksh R48.

As far as the ssh problem with android, this definitely becomes grey-zone material to my knowledge, which is limited in this area. I'm pretty sure android has udev amputated so it lacks a fully working /dev/tty. This could probably be patched over though with some terminal work, mknod action. I've gotten ssh working under "terminal ide", but haven't used the stock ssh stuff in /system/bin. Now that you mention this though, I will try and get stock ssh going as well as openssh which had a similar problem last time I tried (thx for mentioning your support thread, I'll be checking that out).

Here's what I searched to shed a little light on that ssh /dev/tty issue: duckduckgo.com/html5/?q=ssh+%27dev%2Ftty%27+error.
It looks like you've done a ton of searching though.

For su commands, I would try using, e.g.,
...opposed to "su -c", or better e.g.,
...and see if that gives any different results.
I just know from experience "su -c" hasn't always cut it.


*edit: Couple other things worth trying:
1) preappend command with environment variable: global TTY=/dev/pts/0 ...
2) use the -T option when calling mksh, i,e. ssh ... -e mksh -T /dev/pts/0 ...

E:V:A said:

@7175 : Hey thanks for looking into this. I will definitely try your suggestions.
However, I've already tried the -T flag to ssh, but never by specifying the pts to use, as it should be dynamically allocated by the ptmx. (As it could be already in use, by something else, like ATE.) Also, what does "global" do?

You also mentioned "stock ssh" in /system/bin. I've never seen that. Perhaps that could be a better solution, unless its' even more crippled. Does it also support sshd (i.e. ssh -D)?

In the mean time, is there any particular reason why you decided to dynamically link mksh? (I assumed it would have been more portable with static linking?)

I see that you did a huge amount of work on that mkshrc script. Looks crazy (in a good way)!

Finally, and aside, I also noticed that when I start mksh from ssh session, there are some aliases in there, that is not specified in the mkshrc, nor the .profile of my SSH server, so where the heck do they come from?

Code:

[SIZE=2]u0_a202@MSM8960:home # alias
autoload='typeset -fu'
functions='typeset -f'
hash='alias -t'
history='fc -l'
integer='typeset -i'
local=typeset
login='exec login'
nameref='typeset -n'
nohup='nohup '
r='fc -e -'
source='PATH=$PATH:. command .'
type='whence -v'[/SIZE]

Perhaps they're hardcoded or sourced from somewhere? Have you seen this?

Click to expand...

Click to collapse

Thanks man lol it's a big rc. It was 50KB before I converted a bunch of the functions to fpath functions in /etc/fpath and stripped most of the comments. It should give a pretty quick shell in terminal though with the ulimit and renicing tweaks. The PS1 prompt is pretty awesome, probably the best thing in the mkshrc I included.

"global" is a function just like "typeset" or "declare" in bash. The difference is that global prevents an accessed variable from being localized/ different than the global value in a function. This is a mksh function, bash will use "declare -g". There are a couple of example usage in the mkshrc, like with the set -x spam fix for the command prompt.

I checked and stock ssh in /system/bin has the "-D" flag. I would also check rolo's zshaolin zsh project which comes with some afterstock ssh stuff, ssh keygen, ssh keysearch, different ssh etc.

Yeah I just symlinked sh to save space really and for portability to try out other shells like bash and zsh(this works!) to run the system. Recently I switched to using a copy of mksh as sh since "chattr +i" wasn't protecting the symlink from being over-linked.

The aliases you posted are in fact hard-coded. If you do "strings /system/bin/mksh | grep -C7 autoload" or something similar, you'll see them in the binary. I always wondered where those came from too until I read through the mksh manual over a couple months, that's some pretty dense reading lol. mksh is a superb shell though, so it's definitely well worth it.

Sorry I can't be of more help with the ssh /dev/tty problem. I will keep my eyes open though, and try not to be so lazy using telnet for all my remote connections. I mean that's basically using the internet without a condom lol.


(*Also, just something I should mention about the install zip contents: not all the /etc/fpath functions are finished yet. I still gotta tweak the ssh ports from /etc/ssh and finish my xargs. Most should work though and are safe and won't interfer with shell usage, even if autoloaded.)

@E:V:A : Glad the first execution you mentioned worked. I would try messing with quotes/double quotes around the shell command, i.e. ssh ... -T su -c 'exec ...' or try ssh ... -T su -c exec "...", as something to test out further. This can make a lot of difference.

The ulimit shell function can control all sorts of performance related stuff, priority, max niceness, etc. You can see all values with "ulimit -a".

I understand your hesitance about the libc.so replacement. I would keep a copy of the old one and twrp backup before trying out a new one. I get what you're saying about statically linked binaries. I just fixed my i7 desktop that had a blown power supply, so I'll be looking into this. Thanks for the linaro link!

@mirabilos : Thanks for taking a look at this and providing us with mksh! I updated my OP with the specifics about the rebinds for "control I" and "control U" that you mentioned, and I added a suffix to the beginning comments of the file. Something like:
Not sure if that's what you wanted me to add in. I included an extra zip to download in the 2nd post that contains the official mkshrc 2014/01/11, my personal mkshrc, and a "diff -ur" patch (also attached to this post).

I will look into getting a statically linked mksh compiled, that sounds like something really handy to keep around.

edit: Here's basically what I added to the beginning and end of PS1 to trim down its "set -x" verbosity to 3 lines. I'd like to get it down more but haven't figured it out.

@mirabilos : Thank you for your detailed replies, and I would be more than happy to contribute if you see anything from the content I posted.

It is interesting to see the other possibilities for turning off xtrace within the PS1. I didn't even realize you could set positional parameters after the set -x... I think I like that one the most. It just seems more array-like and streamlined, although I guess the same number of variables is still used. I look forward to seeing what updates you'll have on the exit trap and localization. This is something I'd like to toy around with.

I took your advice and set out to get a mksh static built. I ended up compiling with GCC, running "LDSTATIC=-static ./Build.sh" in the mksh source directory. I'm pretty new to cross compiling, so I didn't really get that far or try that hard before chrooting to a fresh Ubuntu terminal on my Nexus, installing GCC, and compiling there. The only thing I changed in the source code was ${ENV:-~/.mkshrc} to ${ENV:-/system/etc/mkshrc} in the "main.c" source file. I can include this in the OP though as you mentioned before. (edit: just saw this as a compiler option with Build.sh)

Edit: I see the option now in the Build.sh for changing the mkshrc path. Also, got R50 rebuilt static with klibc. Damn that got the size way down from using glibc, 684K to 204K.

alanthehat said:

R50e is released -- see https://www.mirbsd.org/mksh.htm#clog

Click to expand...

Click to collapse

Thanks for the heads up. Updated the binaries posted in the 2nd OP to mksh R50e.

Reinkaos said:

Hi @7175, good stuffs here man :good:
I've been using your modded mkshrc, and I can see a hell of a lot of work you put into it. Looks really cool. Plus it's really a good place to learn for someone like me, I definitely had learnt a lot for the past few days. Tons of aliases too, and I'm getting used to using it now. I really appreciate your stuffs here, hope to see more of it

Click to expand...

Click to collapse

Thanks for the feedback man! Glad it's been a good learning experience like it has been for me.