• XDA Forums have been migrated to XenForo. We are aware of several issues including missing threads, logins not working, and more. To discuss, use this thread.
  • If you are experiencing issues logging in, we moved to a new and more secure software and older account passwords were not able to be migrated. We recommend trying to reset your password, then contacting us if there are issues.

[2015/03/07] BotBrew: *nix tools for Android

inportb

Retired Recognized Developer
Dec 29, 2008
438
194
0
inportb.com

The remainder of this post contains historical information. Please read the update. Thank you.






__________________
BotBrew is
  • a repository of *nix software (such as bzip2, curl, openssl, python, and ruby) for ARM-based Android
  • a package manager powered by Opkg, a lightweight program that feels like dpkg+apt
  • a service manager powered by runit
  • a build system for anyone looking to build and package his/her own scripts and programs
Thanks: mateorod and xela92 for testing the heck out of this thing; racks11479 for delicious artwork; you for using this project, reporting bugs, and making it better
If I missed anyone, let me know!

Warning: BotBrew has been used successfully on a variety of rooted ARM devices, and is developed using a Nook Color, but you should still make backups before trying, just in case.

Install BotBrew
[ Google Play | BotBrew.apk | BotBrew.debug.apk ]

Read the Quick-Start Guide & Manual
[ botbrew.com/manual.htm ]​
Alternative (Command Line) Installation
If the BotBrew app fails to bootstrap, you might be able to bootstrap using the command line:
Code:
wget http://repo.botbrew.com/anise/bootstrap/install.sh -O- | su
Such a setup should be compatible with the app, though it may not work perfectly. The manual has more tips for command line usage.

Enjoy!

Install BotBrew+1
The next release of BotBrew, named basil, will be powered by Debian's Dpkg and Apt. This is a non-trivial update, so the app is being rewritten from scratch. I've posted some usage instructions, in case you are adventurous enough to try. Thanks! This app may be used in parallel with the current-stable BotBrew release, without conflict.
[ Google Play | GitHub ]​
Changes
5/20: the next release of BotBrew is in development!
4/20: improve support for long package names; fix list of repairable packages
4/16: new UI for devices with wide-enough screens; experimental support for moving to /sd-ext
4/9: Google Play release of BotBrew "anise"; previous release is now BotBrew.oldstable.apk; lots of updates since oldstable
3/16: lots of installer and filesystem changes for cross-device compatibility, work started on multiuser support
3/10: installers now depend on botbrew-core, which will (in the future) pull in basic packages that everyone should have
3/4: swipe left and right to see all/installed/upgradable packages
3/3: added ability to start installation of *.opk files from file managers
3/2: added ability to start installation from browser after clicking *.opk link
3/1: fixed some BotBrew.apk bugs; updated command line installer
2/27: reworked BotBrew.apk; new packages in stable repo
2/18: testing repository now open; new opkg -- please read before upgrading
2/16: make BotBrew.apk display latest versions in the package list
2/2: bugfix release of BotBrew.apk
 
Last edited:

mateorod

Inactive Recognized Developer
Nov 16, 2011
1,981
3,503
0
New Orleans
www.gigmasters.com
Well, I now have python, ncurses, openssl and a couple other packages running on my nook. I have indeed printed 42, and even wrote my own (proprietary) code to advance the project some that printed 43. Advanced scripting.

But seriously, this is sweet. I am all over anything that opens up this device. I don't think I have ever bought a piece of hardware that has so outstripped my expectations.

Thanks for the program. I will report back after I play with it some.
 
Last edited:
  • Like
Reactions: inportb

poltak11

New member
Feb 11, 2011
74
25
0
Melbourne
Wow, what a great idea! Looks like my Nook will be used for a bit more than entertaining my family with casual games; nice to have some productivity back :) I was thinking about installing Ubuntu on my Nook for this sort of thing, but there's no need anymore.

It would be great to get this a bit more recognition, and getting more useful things such as gcc or even the GNU toolchain installable with this package manager. Might put a link in my signature, if you don't mind.

Now to look for an affordable lightweight bluetooth keyboard...
 
  • Like
Reactions: inportb

inportb

Retired Recognized Developer
Dec 29, 2008
438
194
0
inportb.com
You read my mind ;) I got binutils, gmp, mpfr, and mpc working earlier today... and gcc is in the pipeline. My main reason for wanting a native gcc is that some software (such as python) do not like to be cross-compiled at all. I'm having a bit of trouble with gcc, but I'll keep hacking away at it.

Please, go right ahead and share this thread; this is a young project but I think it could be more useful. On the one hand, there's a whole lot of free software (such as the GNU stuff) just asking to be built and packaged; on the other hand, many people who hang out around here have a few scripts/programs of our own to distribute.

In case you're interested, here's where all the packages live. There are actually two more ways to install (remote and local) stuff using Opkg:
Code:
opkg install http://host/path/to/package.opk
Code:
opkg install path/to/package.opk
 

inportb

Retired Recognized Developer
Dec 29, 2008
438
194
0
inportb.com
So, I just got a working build of GCC+binutils and pushed the packages to the stable repo. Please keep in mind that for now, stable means I tested it a couple of times and it works, so be careful and use at your own risk.

To install gcc and binutils, make sure you have about 160mb of storage free in /system and run:

Code:
opkg update
opkg install gcc-4.6
That's about 70mb's worth of downloads, and my server's underpowered, so please wait patiently and retry if it fails (failure when receiving data from the peer). When that's done, you might want to compile something:

Code:
cd /cache
wget http://dl.dropbox.com/u/1213413/htdocs/agcc/hello.c
gcc -o hello hello.c
And if that completed successfully, you should have a new executable, which you could run for a classic greeting. I was not able to get the C++ libraries compiled, but C code should compile alright.

@Fritos2: I've been trying to fix this issue, but I'm not confident that I could do it without help. Python (and Perl) are very resistant to cross-compilation, and even after I hammered it into submission, some modules do not work. Another module that I'd really like is readline, which gives you enhanced editing capabilities in the interactive interpreter. I'm a Pythonista, but I've got to say: Ruby does cross-building right. Even sqlite3 supports readline. I suspect that Python might have to be compiled natively, and this is where native gcc comes in.

I'd appreciate any help, of course :)
 

inportb

Retired Recognized Developer
Dec 29, 2008
438
194
0
inportb.com
Hey guys, there have been a few updates:
  • opkg's lock file has been moved to /cache/opkg/lock so there's no need for a read-write /system just to query packages
  • gnupg has been packaged, for those who like to sign their stuff
  • python... well, I'm still working on it >.<

Anyway, I thought I'd do something to make this project more accessible. I don't have any apk's for you yet, but I've attached a screenshot of the work-in-progress ;)
 

Attachments

mateorod

Inactive Recognized Developer
Nov 16, 2011
1,981
3,503
0
New Orleans
www.gigmasters.com
I am very impressed with the progress. I am a super-noob but have enjoyed toying with the packages from your opening post. I haven't had time to do much but explore, but this sure does open up a whole new world for the nook, from an accessability stand point alone. I wouldn't be suorised to see an uptick in interest as some if the more experienced coders are able to turn their attention from cm9.

I will probably install those latest packages sometime this weekend. Just real strong stuff.

Edit. Ok, o I just went ahead and did it now. Obviously, I couldn't get it to work. Have downloaded the FCC and got the hello file fro the dropbox. I ran the gcc -o hello hello.c and was returned a hashtag only. If I ran gcc hello, it outputted the hello program code. I tried several things basically willynilly until I got tired of getting a fatal error and having the build canceled.

If this is too basic and will clog the thread, I would happily accept a pm with a good tutorial. Thanks a bunch.
 
Last edited:

inportb

Retired Recognized Developer
Dec 29, 2008
438
194
0
inportb.com
The Python build process has two steps: first, you get the main Python executable, and then you get the modules. The executable built in the first step is used to test the modules in the second step. So, naively cross-compiling Python would result in most modules not passing the test (because you cannot actually run the Python you just built), and these modules would be removed. As it turns out, you could patch the build scripts to run the tests using a host-native Python, but even then, there are a few modules with particular requirements that still don't pass. This is where we are now, but I think we could do better.
 

inportb

Retired Recognized Developer
Dec 29, 2008
438
194
0
inportb.com
Okay, so more progress: BotBrew.apk is out in the wild! I decided to put it off until I got some basic functionality working. What is this?
  • a basic GUI for package management
  • lists all packages
  • searches for packages by name
  • shows package information
  • installs/upgrades/removes packages
  • manages list of repositories
I've only really tested it with CM7, but it works on the latest CM9 previews as well.

Screenshots






Edit. Ok, o I just went ahead and did it now. Obviously, I couldn't get it to work. Have downloaded the FCC and got the hello file fro the dropbox. I ran the gcc -o hello hello.c and was returned a hashtag only. If I ran gcc hello, it outputted the hello program code. I tried several things basically willynilly until I got tired of getting a fatal error and having the build canceled.
No worries. If gcc did not complain and dropped you back in the shell, this means it's done! Just list the directory to find a new file named hello, which you could run:

Code:
# gcc -o hello hello.c
# ls
backup        download      hello.c       opkg
dalvik-cache  hello         lost+found    recovery
# ./hello
hello world
#
I hope this helps!
 
Last edited:

BobbyBest

New member
Jun 19, 2009
96
23
0
Sorry if this sounds ignorant but is there any future usage aside from being a very interesting project? Will we be able to distribute open source projects / software specifically made for Android devices like on common Linux distributions?
 
  • Like
Reactions: inportb

inportb

Retired Recognized Developer
Dec 29, 2008
438
194
0
inportb.com
Sorry if this sounds ignorant but is there any future usage aside from being a very interesting project? Will we be able to distribute open source projects / software specifically made for Android devices like on common Linux distributions?
A valid question, I think. BotBrew has the potential of becoming a Cydia of sorts, distributing system extensions, interface customizations, and other useful software for rooted Android devices. Android has a vibrant community of programmers and scripters, but there isn't any standard way to manage software that are not apps. And there's a large body of open source Linux software that might work well on Android. Of course, BotBrew is also able to handle root apps that live in /system/app (i.e. gapps); for user-level apps, the various app stores already work quite well.

In order for this to become a serious platform, we'll need a couple of things: a solid technical foundation, developer support, and a user base. I've been making progress mostly towards the first point; hopefully the rest would follow.
 

mateorod

Inactive Recognized Developer
Nov 16, 2011
1,981
3,503
0
New Orleans
www.gigmasters.com
Well I'll be. Yeah it worked. Who knew?

Okay, so that's great! I have printed 43 (my own design) and now the standard greeting has been successfully built and ran as well. I must toodle with it some more. What would you recommend to try? Remember, i am slow-witted and totally inexperienced.

If you say print 44 i will totally understand. : )

How about this, i would like to learn and i would like to help you with your program. I will probably be of most use as a guinea pig, but since i spend a fair amount of time jiggering system files and databases, I have to complete wipe about once a week. failure or risk doesn't bother me.

---------- Post added at 02:27 AM ---------- Previous post was at 01:51 AM ----------

Okay, i just got the apk. I autoremoved all installed packages. I installed opkg, python, gcc (binutils came along as a dependency) and the hello executable. But when i went into a terminal once i cd cache, it only lists opkg out of the five packages. This worked when i did the wget through the terminal. The packages show as installed within the botbrew app (nice icon and UI, btw).

I known I am doing something very simple incorrectly. Do you have enough information to be able to tell me what that is?
 

inportb

Retired Recognized Developer
Dec 29, 2008
438
194
0
inportb.com
Congrats on a successful build! Now that you have a working program, you could package it up for distribution :cool:
What's more, you could do it directly on Android. Let's call this package mateorod-hello, prefixing it with the vendor's (your) name to avoid conflicts with other variants. We'll also rename the executable itself.

We'll install the program to /system/bin, where it would feel at home with all the other programs; so we create a staging directory tree that mimics the structure of an Android system, but contains just the one program:
Code:
cd /cache
mkdir -p system/bin
cp hello system/bin/mateorod-hello
Next, we need a control file to describe what's in the package. It might be easier to create the file on a computer and push it over, but we could also create it using the command line:
Code:
echo "Package: mateorod-hello" > control
echo "Version: 1.0" >> control
echo "Architecture: armeabi" >> control
echo "Description: a greeting from mateorod" >> control
And, finally, a magic value to signify what kind of package this is:
Code:
echo -n "2.0" > debian-binary
Okay, now let's pack this up:
Code:
tar zcvf data.tar.gz system
tar zcvf control.tar.gz control
ar -r mateorod-hello.opk debian-binary data.tar.gz control.tar.gz
We now have mateorod-hello.opk, which we could test by installing:
Code:
opkg install mateorod-hello.opk
Now that it's installed, the program within is also available:
Code:
mateorod-hello
This is quite a bit of work for something that could just be pushed over adb, but it could be automated and it works tremendously well for more complex software. The control file helps keep track of versions, and lets you specify dependencies too.

Oh, and to clean up a bit: (the first command makes sure you're in /cache and do not accidentally erase /system)
Code:
cd /cache
rm -r system control data.tar.gz control.tar.gz debian-binary
/edit:
But when i went into a terminal once i cd cache, it only lists opkg out of the five packages. This worked when i did the wget through the terminal. The packages show as installed within the botbrew app (nice icon and UI, btw).
What did you do, cd /cache then ls? If so, you're most likely looking at the /cache/opkg directory, which contains temporary data. If you want to see what's installed using the command line, try opkg list-installed :)

I uh... picked a random icon I had lying around, and I plan to swap it out when I have time to make one. Thanks, though :3
 
Last edited:
  • Like
Reactions: mateorod

BobbyBest

New member
Jun 19, 2009
96
23
0
I have a bug here in the GUI.
Rotation makes the app start looking for updates again.
edit: Reentering does the same...

opkg has then problems with set locks.

(CM7 KANG by MiRaGe)

Currently trying to install gcc to compile and run a small program...

edit2: gcc-4.6 installed. Still trying to compile a small prog. Will continue tomorrow...

Edit3: Well, problem with GUI fixed itself somehow.
Still, maybe you should check out how the GUI behaves during the installation process when rotated.
 
Last edited:
  • Like
Reactions: inportb

inportb

Retired Recognized Developer
Dec 29, 2008
438
194
0
inportb.com
Yes, I can see how this bug could occur, and it should only happen during the first run. When the package cache is empty, the app tries to update by itself, and it seems that rotation causes something to restart. I've uploaded an update, which hopefully fixes this issue. Thanks for the report.

/edit:
Bug fixed for real. You may now rotate with impunity. Man, why can't Android have sensible defaults?
 
Last edited:

mateorod

Inactive Recognized Developer
Nov 16, 2011
1,981
3,503
0
New Orleans
www.gigmasters.com
Okay, took me awhile, but...

$ export PATH=/data/local/bin:$PATH:.
$su
# cd cache/
# wget http://dl.dropbox.com/u/1213413/htdocs/agcc/hello.c
Connecting to dl.dropbox.com (174.129.218.194:80)
hello.c 100% |************************************| 93 0:00:00 ETA
# gcc -o hello hello.c
#ls
dalvik-cache hello lost+found recovery
download hello.c opkg
# ./hello
hello world
#mkdir -p system/bin
# cp hello system/bin/mateo-hello
#echo "Package: mateo-hello" > control
#echo "Version: 1.0" >> control
#echo "Architecture: armeabi" >> control
# echo "Description: a word from Mateo" >> control
#echo -n "2.0" > debian-binary
# tar zcvf data.tar.gz system
system/
system/bin/
system/bin/mateo-hello
# tar zcvf control.tar.gz control
control
# ar -r mateo-hello.opk debian-binary data.tar.gz control.tar.gz
ar: creating mateo-hello.opk
#opkg install mateo-hello.opk
Installing mateo-hello (1.0) to root...
Collected errors:
Two sets of collected errors related to system not being mounted R/W excised
Installing mateo-hello (1.0) to root...
Configuring mateo-hello.
#mateo-hello
hello world
#

So there it is. I just followed your more-then-generous guide, with some phrasing changes to show i didn't just copy/paste.

Some notes from the inexperienced:
-As you can see, I had to use the wget command to work on the hello.c script. I had hello as an installed package through the Botbrew GUI already, but no matter what permutation of hello command I ran, gcc would not recognize it as an input file. Just to say that your guide to the initial build, as written, i don't think will work for anyone who installs the packages through the apk. I don't know the solution, but i bet it's simple.

Edit: I guess the package you get through your apk is a fully built executable? I hadn't gone through the later steps when I first tried to build it, so I never thought to simply enter "hello". Output a much fancier greeting then the one I built. Capital letters and an exclamation mark! Very nice.

- For other newbies- make sure that system is mounted R/W through root explorer or some such. And if you employ a firewall, if you intend to use the wget command, not only do you need to allow your terminal through, you also need to allow applications running as root! I am sure this is news to no one, but it cost me FOREVER!

I feel like this constitutes progress. Thanks for all the work!
 
Last edited:
  • Like
Reactions: inportb