[TOOLS][ZIPS][SCRIPTS] osm0sis' Odds and Ends [Multiple Devices/Platforms]

Captain_Throwback

Recognized Developer
Aug 22, 2008
20,108
22,393
203
The Nothing
Didn't try it yet since they changed the adb source from C to C++. Figured I'd see if I could get 1.0.31 to compile first since it's in C and then weigh between the options of compiling the now primarily cpp based 1.0.32 adb with g++ or to convert it back to C and use gcc (which I'm more comfortable with).

Other than that, I have the crypto, ldl, zip, zlib, and other dependencies statically built in. So it's just a matter of compiling in C++ (unless there are added dependencies in 1.0.32? I glanced over it and only saw a few different things in the Android.mk file but I didn't look closely)
Did you ever get the newer static adb built? I'm still looking for this and hoping someone can make it happen :).
 

Surge1223

Recognized Contributor
Nov 6, 2012
2,603
7,395
203
Florida
Did you ever get the newer static adb built? I'm still looking for this and hoping someone can make it happen :).
Ironically I've been working on this lately. Compiling the dependencies between classes lol. When I get home, maybe I'll watch the ending of The Last of the Mohicans, or something and get in the zone and knock out a new adb lolol.

Sent from my Nexus 6P using XDA-Developers mobile app
 

osm0sis

Senior Recognized Developer / Recognized Contribut
Mar 14, 2012
14,347
32,235
263
Halifax
Did you ever get the newer static adb built? I'm still looking for this and hoping someone can make it happen :).
Ironically I've been working on this lately. Compiling the dependencies between classes lol. When I get home, maybe I'll watch the ending of The Last of the Mohicans, or something and get in the zone and knock out a new adb lolol.
Yeah! I also thought it was funny that I'd only just asked you about this on Hangouts like 3 hours before Captain's post. You're on our minds man! :laugh::highfive:
 
  • Like
Reactions: Surge1223

mikeoswego

Senior Member
May 4, 2014
903
773
0
Northern Indiana
I had some time to play around with gappsintegrator on a Moto X Pure with the stock 6.0 rom, systemless root and Open GApps Super package installed. I found that for some reason, getprop wasn't working while running from /su/su.d. This caused the script to attempt to process the upgraded GApps using the KitKat rules since $sdkver was less than 20. Since the du format was wrong for the free space check, the script would fail indicating no free space.
This change fixes that:
Code:
Original:
# get SDK version to perform different actions due to /data/app layout changes 
sdkver=`getprop ro.build.version.sdk`;

New:
# get SDK version to perform different actions due to /data/app layout changes 
sdkver=`grep "ro.build.version.sdk" /system/build.prop | cut -d\= -f2`;

# abort if failed to get SDK version
if [ ! "$sdkver" ]; then
   abort=1;
   logbuff echo -ne "\n\x20* -- SDK version unknown";
fi;
As a cosmetic change since /su/su.d is an option:
Code:
Original:
# chmod -R 755 /system/etc/init.d /system/su.d

New:
# chmod -R 755 /system/etc/init.d /system/su.d /su/su.d
Finally, an app that was notbeing optimized in the Super GApps package was Google Earth and with the Motorola rom, some of their apps like their camera, gallery, Moto Voice are updated from the play store; so I made this change:
Code:
Original:
for i in $(ls /data/app/ | grep -E '^com.android|^com.google.android|^com.chrome'); do

New:
for i in $(ls /data/app/ | grep -E '^com.android|^com.google.android|^com.google.earth|^com.chrome|^com.motorola'); do
Interestingly, SafetyNet still passes on this ROM after integrating the GApps (I also had some things in /system/xbin before discovering /su/xbin and SafetyNet didn't care about that either.)

Last comment: The forcerebuild for apps like Calendar and keyboard/inputmethod apps where runtime-permissions.xml is deleted causes all apps that need location permission to require that to be set again, some apps like Call Recorder by Skavalex lost all their permissions; yet other apps seemed to have their permissions restored from somewhere (like Google keyboard.) Anyway, not a deal breaker, just thought I'd mention it.

Thanks for your work on this, it saves almost 500mb in data after integration with my set of GApps and Moto Apps!
 
  • Like
Reactions: domicq and osm0sis

Surge1223

Recognized Contributor
Nov 6, 2012
2,603
7,395
203
Florida
Did you ever get the newer static adb built? I'm still looking for this and hoping someone can make it happen :).
So at this point it became easier for me to port the aosp build system to arm, since I would have to recompile a lot of libs for newer dependencies.

I've hit a bit of a snag and not sure how to proceed, android switched to clang not to long ago and depreciated alot of their ndk support libs, like stlport no longer exports the includes needed for streams. This is solveable, but requires removing a few namespaces from some source files.

If anyone has any idea how I can get ios stream includes built, by all means let me know.

I did manage to get the newest adb source buildadble on arm though, granted it doesn't produce a binary because of the undefined references.

on the brightside, I did however completely rebuild libc and it now it seems like i have a full static and shared version of what aosp so poorly builds, libc.so is one of the most annoying libs in android, usually because the 3 morons running the NDK for google dont know how to compile right. Yes thats a a very harsh claim, but seriously ever release of the ndk butchers or messes up libc in someway, whether its because they dont know how to make shared libraries properly (as is kinda evident by their cflags and ldflags passed) , or that they dont care is up for debate lol.




Sent from my Nexus 6P using XDA-Developers mobile app
 
Last edited:

bigsupersquid

Senior Member
Sep 22, 2010
2,249
1,666
193
BFE, MO
So at this point it became easier for me to port the aosp build system to arm, since I would have to recompile a lot of libs for newer dependencies.

I've hit a bit of a snag and not sure how to proceed, android switched to clang not to long ago and depreciated alot of their ndk support libs, like stlport no longer exports the includes needed for streams. This is solveable, but requires removing a few namespaces from some source files.

If anyone has any idea how I can get ios stream includes built, by all means let me know.

I did manage to get the newest adb source buildadble on arm though, granted it doesn't produce a binary because of the undefined references.

on the brightside, I did however completely rebuild libc and it now it seems like i have a full static and shared version of what aosp so poorly builds, libc.so is one of the most annoying libs in android, usually because the 3 morons running the NDK for google dont know how to compile right. Yes thats a a very harsh claim, but seriously ever release of the ndk butchers or messes up libc in someway, whether its because they dont know how to make shared libraries properly (as is kinda evident by their cflags and ldflags passed) , or that they dont care is up for debate lol.
Can you share your android_build directory fixes (and any other critical patches you noticed) for arm host? I've been intermittently messing around for years trying to get that working more than the little bit I've had success with.
I'd be more than happy to continue messing around with arm building and maybe I'd even be lucky enough to stumble across the fixes you need in the process.
 
Last edited:
  • Like
Reactions: osm0sis

Surge1223

Recognized Contributor
Nov 6, 2012
2,603
7,395
203
Florida
Can you share your android_build directory fixes (and any other critical patches you noticed) for arm host? I've been intermittently messing around for years trying to get that working more than the little bit I've had success with.
I'd be more than happy to continue messing around with arm building and maybe I'd even be lucky enough to stumble across the fixes you need in the process.
Yeah definitely. Yeah it was a bit of of mess, but since I had python and gcc already compiled for arm, i thought I might as well try to build natively. It can definitely use a lot of fixes, I've been working on it on and off, mostly because I wanted to see if i could compile aapt from my device statically. (just achieved that today).

there aren't very many good resources on how to do all this so I'd be happy to post it up on git.



Sent from my Nexus 6P using XDA-Developers mobile app
 

osm0sis

Senior Recognized Developer / Recognized Contribut
Mar 14, 2012
14,347
32,235
263
Halifax
UPDATE-Nexus.BootUnlocker works for nexus 5x too?
Only to toggle `fastboot oem unlock`, as per the blurb in the OP.

I had some time to play around with gappsintegrator on a Moto X Pure with the stock 6.0 rom, systemless root and Open GApps Super package installed. I found that for some reason, getprop wasn't working while running from /su/su.d. This caused the script to attempt to process the upgraded GApps using the KitKat rules since $sdkver was less than 20. Since the du format was wrong for the free space check, the script would fail indicating no free space.
This change fixes that:
Code:
Original:
# get SDK version to perform different actions due to /data/app layout changes 
sdkver=`getprop ro.build.version.sdk`;

New:
# get SDK version to perform different actions due to /data/app layout changes 
sdkver=`grep "ro.build.version.sdk" /system/build.prop | cut -d\= -f2`;

# abort if failed to get SDK version
if [ ! "$sdkver" ]; then
   abort=1;
   logbuff echo -ne "\n\x20* -- SDK version unknown";
fi;
As a cosmetic change since /su/su.d is an option:
Code:
Original:
# chmod -R 755 /system/etc/init.d /system/su.d

New:
# chmod -R 755 /system/etc/init.d /system/su.d /su/su.d
Finally, an app that was notbeing optimized in the Super GApps package was Google Earth and with the Motorola rom, some of their apps like their camera, gallery, Moto Voice are updated from the play store; so I made this change:
Code:
Original:
for i in $(ls /data/app/ | grep -E '^com.android|^com.google.android|^com.chrome'); do

New:
for i in $(ls /data/app/ | grep -E '^com.android|^com.google.android|^com.google.earth|^com.chrome|^com.motorola'); do
Interestingly, SafetyNet still passes on this ROM after integrating the GApps (I also had some things in /system/xbin before discovering /su/xbin and SafetyNet didn't care about that either.)

Last comment: The forcerebuild for apps like Calendar and keyboard/inputmethod apps where runtime-permissions.xml is deleted causes all apps that need location permission to require that to be set again, some apps like Call Recorder by Skavalex lost all their permissions; yet other apps seemed to have their permissions restored from somewhere (like Google keyboard.) Anyway, not a deal breaker, just thought I'd mention it.

Thanks for your work on this, it saves almost 500mb in data after integration with my set of GApps and Moto Apps!
Looks good! Can you open a pull request for those changes?
 

osm0sis

Senior Recognized Developer / Recognized Contribut
Mar 14, 2012
14,347
32,235
263
Halifax
Here's a script to check and increase the size of su.img if necessary. I've bolded the additional script in comparison to my previously posted /su mounting commands.

Code:
[b]payload_size_check() {
  reqSizeM=0;
  for entry in $(unzip -l "[email protected]" 2>/dev/null | tail -n +4 | awk '{ print $1 }'); do
    test $entry != "--------" && reqSizeM=$((reqSizeM + entry)) || break;
  done;
  test $reqSizeM -lt 1048576 && reqSizeM=1 || reqSizeM=$((reqSizeM / 1048576));
}
target_size_check() {
  curBlocks=`e2fsck -n $1 2>/dev/null | cut -d, -f3 | cut -d\  -f2`;
  curUsedM=$((`echo "$curBlocks" | cut -d/ -f1` * 4 / 1024));
  curSizeM=$((`echo "$curBlocks" | cut -d/ -f2` * 4 / 1024));
  curFreeM=$((curSizeM - curUsedM));
}[/b]

ui_print "Mounting...";
ui_print " ";
[B]umount /system 2>/dev/null;
mount -o ro -t auto /system;[/B]
mount /data;
mount /cache;
suimg=$(ls /cache/su.img /data/su.img 2>/dev/null);
if [ "$suimg" ]; then
  umount /su;

[b]  payload_size_check "$ZIPFILE" "bin/*" "xbin/*";
  target_size_check $suimg;
  if [ "$reqSizeM" -gt "$curFreeM" ]; then
    suNewSizeM=$((((reqSizeM + curUsedM) / 32 + 1) * 32));
    ui_print 'Resizing su.img to '"$suNewSizeM"'M...';
    ui_print " ";
    e2fsck -yf $suimg;
    resize2fs $suimg "$suNewSizeM"M;
  fi;[/b]

  test ! -e /su && mkdir /su;
  mount -t ext4 -o rw,noatime $suimg /su;
  for i in 0 1 2 3 4 5 6 7; do
    case `mount` in
      *" /su "*) break;;
    esac;
    loop=/dev/block/loop$i;
    mknod $loop b 7 $i;
    losetup $loop $suimg;
    mount -t ext4 -o loop $loop /su;
  done;
fi;
The payload_size_check function uses the list entries from unzip for the supplied payload zip and locations within (multiple may be added to the command-line, as shown) to calculate the total required space in mb. It saves this size in the variable reqSizeM for subsequent use. For any wondering, it doesn't use unzip's listed total because I've found (in my gappsintegrator work) that unzip can be unreliable on certain signed zips and APKs, bailing before printing the total, despite reaching the functional end of the archive.

The target_size_check function uses e2fsck to read the used/total block status of a target device or ext4 img then calculates the target's free space in mb. It saves this as curFreeM, as well as saving curUsedM and curSizeM. This can be used with su.img as shown, but also works just as well with block devices in /dev/block in case someone has a mod that can go in /su or /system and wants to make sure there's space for either way. ;)

The script, as written, then adds 32M to whatever the current size of su.img is, to ensure that the installation will fit, and outputs the new size to the user. The "+ 1" in the suNewSizeM calculation could be changed to "+ 2" for a 64M increase, or "+ 3" for 96M, etc., if the mod requires an expansion that large. :cowboy:

The only thing missing is a check that the mod doesn't already exist in su.img before increasing the size of it; easiest way to do that is to check for a file from the mod within /su before making any changes. For a working example, I've just uploaded a new UPDATE-Adreno.Systemless.Installer-signed.zip via the link in the OP. :good:

I work on these projects in my limited time off, so if you like the progress I'm making, or enjoy anything else I've done on xda, please do hit the donate link from my profile. Thanks for your support!

Edit: Added a /system mount because in some recoveries and in FlashFire there is no e2fsck binary without. :good:
 
Last edited:

mikeoswego

Senior Member
May 4, 2014
903
773
0
Northern Indiana
Looks good! Can you open a pull request for those changes?
Done (I think, first time doing this!)

On the deletion of runtime-permissions.xml, did you ever try just removing the lines referencing the integrated app instead of deleting the entire file? If not, I'll experiment with that. Or maybe it would be possible to store a backup and then restore the lines for apps not listed in the rebuilt file.
 
  • Like
Reactions: osm0sis

osm0sis

Senior Recognized Developer / Recognized Contribut
Mar 14, 2012
14,347
32,235
263
Halifax
Done (I think, first time doing this!)

On the deletion of runtime-permissions.xml, did you ever try just removing the lines referencing the integrated app instead of deleting the entire file? If not, I'll experiment with that. Or maybe it would be possible to store a backup and then restore the lines for apps not listed in the rebuilt file.
Nice! Thanks! Noticed you had a GitHub account, so figured you might as well get direct credit for the commit changes. :)

I seem to recall trying it every which way before ending up killing the .xml file altogether, and it was only in forcing the complete rebuild that saved the device from the dreaded System UI crashes. Here's my little write-up where I first introduced it in v8: http://forum.xda-developers.com/showthread.php?p=66716689#post66716689

Edit: Yup, reviewed my Hangouts conversation with @MastahF where we figured it out and more surgical edit attempts showed to be pointless since it's not actually runtime-permissions.xml itself that needed a rebuild, but fully deleting it is the only trigger we could find for whatever process works around those newer APKs causing the app renumbering/breakage.

Feel free to try some more things out though, I don't think I tried rebuilding it and then putting the old one back; you could have it do that and force another reboot directly after only on those blacklisted apps so there is no gap where the rebuilt one gets actively used.. might work. :D
 
Last edited:
  • Like
Reactions: mikeoswego

mikeoswego

Senior Member
May 4, 2014
903
773
0
Northern Indiana
Nice! Thanks! Noticed you had a GitHub account, so figured you might as well get direct credit for the commit changes. :)

I seem to recall trying it every which way before ending up killing the .xml file altogether, and it was only in forcing the complete rebuild that saved the device from the dreaded System UI crashes. Here's my little write-up where I first introduced it in v8: http://forum.xda-developers.com/showthread.php?p=66716689#post66716689

Edit: Yup, reviewed my Hangouts conversation with @MastahF where we figured it out and more surgical edit attempts showed to be pointless since it's not actually runtime-permissions.xml itself that needed a rebuild, but fully deleting it is the only trigger we could find for whatever process works around those newer APKs causing the app renumbering.

Feel free to try some more things out though, I don't think I tried rebuilding it and then putting the old one back; you could have it do that and force another reboot directly after only on those blacklisted apps so there is no gap where the rebuilt one gets actively used.. might work. :D
Thanks, excellent info! So. maybe find all the runtime-permissions.xml files, make backup(s), delete them, wait around for it/them to reappear, restore the backup(s) and force reboot would do the trick. I'll work on that.
 
  • Like
Reactions: osm0sis

skulldreamz

Senior Member
Mar 13, 2012
1,402
1,175
143
Baltimore, MD
When I read these comments, I feel like a little kid listening to the adults talking. Nothing makes sense!! lol

As always, I appreciate what you guys do and share it with the community

Don't stop, all of a sudden pieces will start to fall in place and you'll be like I'm chilling at the adult table, lol. Nah but stick to it and it really does get easier, well some of it, haha :D
@osm0sis = great great great grandfather table ;)
 
  • Like
Reactions: osm0sis

osm0sis

Senior Recognized Developer / Recognized Contribut
Mar 14, 2012
14,347
32,235
263
Halifax
Don't stop, all of a sudden pieces will start to fall in place and you'll be like I'm chilling at the adult table, lol. Nah but stick to it and it really does get easier, well some of it, haha :D
@osm0sis = great great great grandfather table ;)
Haha nah, I picked up most of what I know in the same way, hanging out in kernel threads, talking to devs and tinkering. Also searching and reading a lot. There's still lots (and lots!) of ROM, kernel and app dev stuff I don't know much (or anything!) about, but I know it'll keep making more sense the longer I hang out on xda. :)
 
Last edited:

skulldreamz

Senior Member
Mar 13, 2012
1,402
1,175
143
Baltimore, MD
Haha nah, I picked up most of what I know in the same way, hanging out in kernel threads, talking to devs and tinkering. Also searching and reading a lot. There's still lots (and lots!) of ROM, kernel and app dev stuff I don't know much about, but I know it'll keep making more sense the longer I hang out on xda. :)

Yes, searching and reading is important. I like your threads because I can always learn something from them.

Spoken like a true champ! And to anyone that feels overwhelmed at times, I totally understand. Something that I started doing about 6 months ago was everyday just look up 1 thing for example the syntax of a command or a applet, something small and start a log in a notepad in like Google keep or whatever so you can always go back and benefit from the research you put it in case you forget. Just a idea
 
  • Like
Reactions: mrrocketdog

edu4rd0zs

Senior Member
Jan 8, 2015
51
20
0
Don't stop, all of a sudden pieces will start to fall in place and you'll be like I'm chilling at the adult table, lol. Nah but stick to it and it really does get easier, well some of it, haha :D
@osm0sis = great great great grandfather table ;)
Definitely won't stop. I want to learn and understand more and ultimately compile a ROM and kernel, then share it with the community. I bet it feels good to have it shared all across the world :cool:
 
  • Like
Reactions: skulldreamz