[TOOLS][ZIPS][SCRIPTS] osm0sis' Odds and Ends [Multiple Devices/Platforms]

[shoey63]

This should really be a TWRP native feature.

Do you have the the time (or inclination) to code it, and submit it to Omnirom gerrit?

 

[_that]

Do you have the the time (or inclination) to code it, and submit it to Omnirom gerrit?

I don't want to promise anything, but I've noted it on my personal "ideas for new features" list.

 

[Maxr1998]

This should really be a TWRP native feature.

After reading his post multiple times, I got the understanding that he wants to wipe data except internal storage AND apps (not the backup of the apps), means /data/ excluding /data/media/0/ and /data/app/..

 

[osm0sis]

After reading his post multiple times, I got the understanding that he wants to wipe data except internal storage AND apps (not the backup of the apps), means /data/ excluding /data/media/0/ and /data/app/..

It suggests /data/data should be saved as well but differentiating between system apps and user apps and only saving user app data?

I don't want to promise anything, but I've noted it on my personal "ideas for new features" list.

That'd be a great advanced features for TWRP.

@osm0sis if it is not asking too much you could create a zip that wipe all /data except for the internal memory and user apps? Similar to FlashFire wipe function, allowing TWRP users migrate from rom without the need to backup apps before every time

I don't have a lot of free time at the moment. I've got a few big good things on my todo I'm barely finding enough to pick away at (checking out mike's gappsintegrator updates, etc. - hoping to do so on my days off Sunday or Wednesday). For now, until it maybe becomes a built-in TWRP feature, you could take a stab at it using my Complete Shell Script Zip Reference thread.

 

[KaMyKaSii]

It suggests /data/data should be saved as well but differentiating between system apps and user apps and only saving user app data?

Exactly. The Flashfire already does this but if you are already in TWRP and without installed system ...

I don't have a lot of free time at the moment. I've got a few big good things on my todo I'm barely finding enough to pick away at (checking out mike's gappsintegrator updates, etc. - hoping to do so on my days off Sunday or Wednesday). For now, until it maybe becomes a built-in TWRP feature, you could take a stab at it using my Complete Shell Script Zip Reference thread.

Okay, I understand that to be pretty busy with the amount of scripts you have. And speaking of that, I have a file access problem both on external and internal memory, then used your script to fix permissions and gave it. He should not read the file "/system/lib64/libcutils.so"?

 sdcard Fix Permissions Script
by osm0sis @ xda-developers
 
Mounting...
mount: mounting /dev/block/dm-0 on /data failed: Device or resource busy
 
Setting /data/media to media_rw and fixing...
I:Set overlay: 'lock'
I:TWFunc::Set_Brightness: Setting brightness control to 5
I:TWFunc::Set_Brightness: Setting brightness control to 0
ioctl(): blank: Invalid argument
ioctl(): blank: Invalid argument
I:TWFunc::Set_Brightness: Setting brightness control to 162
I:TWFunc::Set_Brightness: Setting brightness control to 5
I:TWFunc::Set_Brightness: Setting brightness control to 0
ioctl(): blank: Invalid argument
CANNOT LINK EXECUTABLE: "/system/lib/libcutils.so" is 32-bit instead of 64-bit
page record for 0x7f79dec010 was not found (block_size=32)
/tmp/updater: line 47: /system/toolbox: not found
CANNOT LINK EXECUTABLE: "/system/lib/libcutils.so" is 32-bit instead of 64-bit
page record for 0x7f9ca16010 was not found (block_size=32)
 
Setting TWRP directory perms...
 
Unmounting...
 
Done!

 

[osm0sis]

And speaking of that, I have a file access problem both on external and internal memory, then used your script to fix permissions and gave it. He should not read the file "/system/lib64/libcutils.so"?

 sdcard Fix Permissions Script
by osm0sis @ xda-developers
 
Mounting...
mount: mounting /dev/block/dm-0 on /data failed: Device or resource busy
 
Setting /data/media to media_rw and fixing...
I:Set overlay: 'lock'
I:TWFunc::Set_Brightness: Setting brightness control to 5
I:TWFunc::Set_Brightness: Setting brightness control to 0
ioctl(): blank: Invalid argument
ioctl(): blank: Invalid argument
I:TWFunc::Set_Brightness: Setting brightness control to 162
I:TWFunc::Set_Brightness: Setting brightness control to 5
I:TWFunc::Set_Brightness: Setting brightness control to 0
ioctl(): blank: Invalid argument
CANNOT LINK EXECUTABLE: "/system/lib/libcutils.so" is 32-bit instead of 64-bit
page record for 0x7f79dec010 was not found (block_size=32)
/tmp/updater: line 47: /system/toolbox: not found
CANNOT LINK EXECUTABLE: "/system/lib/libcutils.so" is 32-bit instead of 64-bit
page record for 0x7f9ca16010 was not found (block_size=32)
 
Setting TWRP directory perms...
 
Unmounting...
 
Done!

That's this function executing:

restore_con() { 
  for i in /system/bin/toybox /system/toolbox /system/bin/toolbox; do
    LD_LIBRARY_PATH=/system/lib $i restorecon -R $*;
  done;
  restorecon -R $*;
}

So a couple of errors are generally expected when it's cycling through locations. As for the linking, @Chainfire only links /system/lib also, presumably just to accommodate older recoveries, where newer ones (which all 64-bit devices should have) will have things working without needing to use the system one.

I could set the path with lib64 at the front in future updates, and that should work on both architectures, but currently the overall function should be working fine as well because of that final command.

 

[KaMyKaSii]

That's this function executing:


So a couple of errors are generally expected when it's cycling through locations. As for the linking, @Chainfire only links /system/lib also, presumably just to accommodate older recoveries, where newer ones (which all 64-bit devices should have) will have things working without needing to use the system one.

I could set the path with lib64 at the front in future updates, and that should work on both architectures, but currently the overall function should be working fine as well because of that final command.

I think we had a misunderstanding. This log is from TWRP 3.0.2

 

[mikeoswego]

I think we had a misunderstanding. This log is from TWRP 3.0.2

Your TWRP log was created while running the fix permissions script. In the section of the script that osm0sis posted in his reply, it looks in 3 different places for toybox / toolbox. Various phones have these in different places, so the script has to figure out where they are and when it looks in a place where they don't exist, TWRP will generate the not found error you are seeing. So it is normal to see some errors when running this script and others like it.

The last 3 lines of your log show that the script completed successfully.

 

[PiggyFlooper]

@osm0sis
I see you as being a master of scripting. Will you please teach me how to write a separate /sbin/sh script to be ran in recovery that senses the android version, such as equal to or above 5.0 (50) like this-

Call it sense.sh

in updater-script-->
run_program("tools/sense.sh");

#!/sbin/sh

android_version="?"
if [ "android_version" -lt 50 ]; then
cp /tmp/priv-app/NexusLauncherPrebuilt/Nexus LauncherPrebuilt.apk /system/priv-app/NexusLauncherPrebuilt.apk;
else
cp /tmp/priv-app/NexusLauncherPrebuilt/Nexus LauncherPrebuilt.apk /system/priv-app/NexusLauncherPrebuilt/NexusLauncherPrebuilt.apk;
fi;

Please help me to figure out how to write this so it will get android version

 

[_that]

@osm0sis
I see you as being a master of scripting. Will you please teach me how to write a separate /sbin/sh script to be ran in recovery that senses the android version

Take this line from an apprentice of scripting:

apilevel=$(awk -F '=' '/^ro.build.version.sdk/ { print $2 }' /system/build.prop)

Make sure that /system is mounted before.
The list of SDK versions is here -> https://developer.android.com/guide/topics/manifest/uses-sdk-element.html#ApiLevels

 

[osm0sis]

@osm0sis
I see you as being a master of scripting. Will you please teach me how to write a separate /sbin/sh script to be ran in recovery that senses the android version, such as equal to or above 5.0 (50) like this-

Call it sense.sh

in updater-script-->
run_program("tools/sense.sh");

#!/sbin/sh

android_version="?"
if [ "android_version" -lt 50 ]; then
cp /tmp/priv-app/NexusLauncherPrebuilt/Nexus LauncherPrebuilt.apk /system/priv-app/NexusLauncherPrebuilt.apk;
else
cp /tmp/priv-app/NexusLauncherPrebuilt/Nexus LauncherPrebuilt.apk /system/priv-app/NexusLauncherPrebuilt/NexusLauncherPrebuilt.apk;
fi;

Please help me to figure out how to write this so it will get android version

Take this line from an apprentice of scripting:

apilevel=$(awk -F '=' '/^ro.build.version.sdk/ { print $2 }' /system/build.prop)

Make sure that /system is mounted before.
The list of SDK versions is here -> https://developer.android.com/guide/topics/manifest/uses-sdk-element.html#ApiLevels

Yup, @_that is right on. Only thing different I'd suggest would be using grep and cut to find the prop value since CM Recovery doesn't have awk.

I've always used the following:

file_getprop() { grep "^$2" "$1" | cut -d= -f2; }

Then you can invoke it as follows:

apilevel=$(file_getprop /system/build.prop ro.build.version.sdk)

 

[PiggyFlooper]

Yup, @_that is right on. Only thing different I'd suggest would be using grep and cut to find the prop value since CM Recovery doesn't have awk.

I've always used the following:

file_getprop() { grep "^$2" "$1" | cut -d= -f2; }

Then you can invoke it as follows:

apilevel=$(file_getprop /system/build.prop ro.build.version.sdk)

Okay, thank you guys both, I used the method as a backup in case my way doesn't work, either way, what i got is successful- flashing a zip file meant for 5.x+ to a version 4.4x and below- might as well upload what was came up with, it's TBOs Nexus Launcher n wallpaper picker ( the launcher not working in 4.4 )
Wallpaper picker does though

Another question, by using apilevel, does this go by 5.0=50 or 4.4=19, which way applies using apilevel?

Zip- https://db.tt/vQzv3ovo

 

[osm0sis]

Okay, thank you guys both, I used the method as a backup in case my way doesn't work, either way, what i got is successful- flashing a zip file meant for 5.x+ to a version 4.4x and below- might as well upload what was came up with, it's TBOs Nexus Launcher n wallpaper picker ( the launcher not working in 4.4 )
Wallpaper picker does though

Another question, by using apilevel, does this go by 5.0=50 or 4.4=19, which way applies using apilevel?

Zip- https://db.tt/vQzv3ovo

It goes according to the SDK version list @_that linked for you. KitKat = 19, Lollipop = 21-22, Marshmallow = 23. See my gappsintegrator script on GitHub for an example of how to use them.

Speaking of which, I'd better look into @mikeoswego's pull request before I get busy again; I ended up trying to clean install Windows 10 x64 over (my free upgraded) Windows 10 x86 (on a x64 box) yesterday so I could play with Bash on Ubuntu on Windows, but sadly it seems the box is too old for the Windows 10 x64 installer to run correctly. Big time-waster.

 

[mikeoswego]

Speaking of which, I'd better look into @mikeoswego's pull request before I get busy again; I ended up trying to clean install Windows 10 x64 over (my free upgraded) Windows 10 x86 (on a x64 box) yesterday so I could play with Bash on Ubuntu on Windows, but sadly it seems the box is too old for the install to run correctly. Big time-waster.

I am so frustrated with Win10. I have an older Dell M2400 that with Win7 I couldn't use a video driver newer than 275.33 from 2011 (due to a known problem with how Dell configured the Nvidia card.) When I allowed the Win10 upgrade to push onto it, it immediately boot looped due to a newer driver being used and I had to restore Win7. I was finally able to do a clean install on it and figured out how to use that old video driver and set Win10 to NOT try to update it ever and it worked fine. With the anniversary update, it insists on updating the video driver again, boot loops, can't roll back had to do a system image restore. I suppose I could do a clean install with the anniversary update but it is probably time for a new machine. What a PITA. (I have an older ASUS G50V that also wouldn't upgrade to Win10 but a clean install worked fine and the anniversary update was problem free.)

 

[PiggyFlooper]

It goes according to the SDK version list @_that linked for you. KitKat = 19, Lollipop = 21-22, Marshmallow = 23. See my gappsintegrator script on GitHub for an example of how to use them.


Okay, I'll check it out, actually I already have it. So the formula you gave is for sdk version, gotcha, thanks

 

[Paulo Narley]

@osm0sis
I do not quite understand about programming language.
More you, considering the amount of contributions to the community, is certainly someone who dominates it.
That said it would be possible to carry this tool for Linux?
[Tool][Win] GOptimize v1.44 (BASH script for CYGWIN) by @gu5t3r
It would be interesting to see it next to superR's Kitchen of @SuperR.

 

[osm0sis]

@osm0sis
I do not quite understand about programming language.
More you, considering the amount of contributions to the community, is certainly someone who dominates it.
That said it would be possible to carry this tool for Linux?
[Tool][Win] GOptimize v1.44 (BASH script for CYGWIN) by @gu5t3r
It would be interesting to see it next to superR's Kitchen of @SuperR.

Eh, I script and I compile. Sometimes I have ideas.

That tool just received an update back in April, so it's still being maintained. There's no Linux equivalent? If no, Cygwin shell is basically bash, do the scripts not work? You probably just need the right binaries installed on your box. I'd ask the author about it in his thread.

Edit: They actually discuss it on the last page of that thread, so there's your answer from the expert. :good:

 

[Paulo Narley]

Eh, I script and I compile. Sometimes I have ideas.

That tool just received an update back in April, so it's still being maintained. There's no Linux equivalent? If no, Cygwin shell is basically bash, do the scripts not work? You probably just need the right binaries installed on your box. I'd ask the author about it in his thread.

Edit: They actually discuss it on the last page of that thread, so there's your answer from the expert. :good:

OK, thanks for the reply, and thanks for the contributions the community.
I will continue using it in a VM, since I do not have enough knowledge to make the proper associations between the script and the wine.

Thanks again for your attention.

:fingers-crossed::good::highfive:

 

[osm0sis]

A few updates!

First, nano has another official update. :good:

I've also added a new 00-resizesystem.sh addon.d script to the OP to help devices where the /system partition has been resized to hold larger ROM/GApps packages, but the ROM in use has system.dat format zips with no resizing functionality, so the extra space would get lost in a ROM delta flash, breaking things when GApps attempted to restore. My script determines the system partition location on any emmc device, then resizes the system partition to its maximum before the GApps restore occurs.

Next, I'm pleased to report that with the gappsintegrator v9 update all known issues are resolved! Massive thanks to @mikeoswego for his work on the major fixes for the packages.xml entry edits (resolving the longstanding issue of multiple apps redownloading during the integration process - turns out because their entries were getting eaten up with the one prior), and implementing the runtime-permissions.xml backup/restore functionality for the workaround blacklisted apps (so that user apps retain their granted permissions after a forcerebuild), as well as his time testing and problem-solving with me on Hangouts. There is a forced reboot during the restore process so don't be alarmed. :highfive:

I then combined my rough draft version of the runtime-permissions.xml backup/restore with his and fixed a few other outstanding issues like the date being incorrect at times in the log, etc. and I also raised the timeout for writing to sdcard to 900 seconds (15 minutes) since I had reports it was writing to /data/local on older devices.

I work on these projects in my limited time off, so if you like the progress I'm making, or enjoy anything else I've done on xda, please do hit the donate link from my profile. Thanks for your support!

UPDATE-nano.Terminal.Editor.v2.6.3-signed.zip:
- update to latest nano official source.

gappsintegrator: v9 - improve Marshmallow breakage workaround, fix optimization, many fixes/improvements:
- backup/restore runtime-permissions.xml (thanks @mikeoswego) to a specific subdirectory and simplify check/naming
- add single-line packages.xml entry fix (thanks @mikeoswego) to Lollipop, fixing multiple app redownloads during optimizations
- alphabetize /data/app filter for possible future similar OEM stock app additions (ex. Motorola, thanks @mikeoswego), and Marshmallow updated blacklist (thanks @mikeoswego) for breakage workaround
- increase sdcard writability wait limit to 15 minutes to better accomodate slower devices
- rewrite use of busybox for broken/limited toolbox implementations of chmod (thanks @mikeoswego) into a compatibility check using busybox as a fallback
- wait for date to stop being epoch (Jan 1 1970) during boot to record earliest accurate processing start time, and ensure accurate timestamps for filesystem operations
- add basename, cut and dirname to basic script environment checks

https://github.com/osm0sis/gappsintegrator/commit/b1e9b42b48fd3eeba13eadc421c238ba4fd82bc4
also merged/cherry-picked Minor updates and Major updates from @mikeoswego.

Download via GitHub's raw copy of the latest version, or extract it from the repo zip. :good:

AK2 also saw a new minor commit this week following a conversation with @Perseus, and I continue to pick away at a major update for AIK.

 

[mikeoswego]

UPDATE-nano.Terminal.Editor.v2.6.3-signed.zip:
- update to latest nano official source.

gappsintegrator: v9 - improve Marshmallow breakage workaround, fix optimization, many fixes/improvements:
- backup/restore runtime-permissions.xml (thanks @mikeoswego) to a specific subdirectory and simplify check/naming

Thanks for the mentions and for all the work you do on these and other things!