[APP][1.6+] SD Maid - System cleaning tool

Search This thread

2_i

Member
May 22, 2013
30
3
Finding Corpses Takes Forever

I am in a situation of about 300 corpses. In finding them SD Maid runs for 12h+ without conclusion. Any suggestions for a resolution?

Background: This is a small phone, Xperia Tipo Dual, running Android 4.0.4. I squeeze numerous apps into the phone using Link2SD and packing those apps into a second partition on SD card. Unfortunately, the card started developing problems and I had to replace it. In spite of my best efforts, the links to the apps in the 2nd partition did not stick and I need to reinstall those apps. Unfortunately, the apps most often fail to install anew if their corpse files are still there. I can remove them by hand, but it is hard to do that for 300+ apps. I normally used SD Maid for the purpose, but it seems to take the age of Universe to uncover this many corpses. Is that to be expected? My rough guess that the time for corpse identification scales more than quadratically with the number of corpses. Could it be that something else is wrong? I restarted the Maid few times over the couple of last days and it always went into such a prolonged search with this being the longest sweep I have patience for.
 

Dark3n

Recognized Developer
Sep 14, 2007
2,567
5,175
twitter.com
@2_i

1: How do you get to the number 300+ and what was the longest you waited for SD Maid to complete?
2: Where specifically are the corpses located (i guess Link2SD linked from /data/data)?
3: What version of SD Maid are you using?

I've tried to reproduce it and it's indeed not that fast, on my N5 @ 4.4 it took 390ms per Object in /data/data
I shoveled about 2000 trash files into /data/data, it seems to scale linear.
Send me a mail later (I'll go sleep now), i think i can give you tweaked version.
 
Last edited:
  • Like
Reactions: 2_i

2_i

Member
May 22, 2013
30
3
1: How do you get to the number 300+ and what was the longest you waited for SD Maid to complete?

That was the number of apps linked under LINK2SD. When I try to restore them, in 90% of cases the installation fails due to "insufficient storage". (There is plenty of space.) After I find the corpses an delete them, the installation goes forward.

2: Where specifically are the corpses located (i guess Link2SD linked from /data/data)?

In my experience I only need to locate those under /data, not sure whether always under /data/data.

3: What version of SD Maid are you using?

It is a version (Pro) that was current in Play 2 weeks ago - don't want to interrupt the Maid's scan now.

I've tried to reproduce it and it's indeed not that fast, on my N5 @ 4.4 it took 390ms per Object in /data/data

I'm now at 15h+.
 
Last edited:

iPusak Gaoq™

Senior Member
Mar 21, 2012
2,163
1,172
Kuching
That was the number of apps linked under LINK2SD. When I try to restore them, in 90% of cases the installation fails due to "insufficient storage". (There is plenty of space.) After I find the corpses an delete them, the installation goes forward.



In my experience I only need to locate those under /data, not sure whether always under /data/data.



It is a version (Pro) that was current in Play 2 weeks ago - don't want to interrupt the Maid's scan now.



I'm now at 15h+.

Have you ever try using the cleanup features in Link2SD to clean all those corpses file?
"Clean up 2nd partition" and "Clean up dalvik-cache" in Link2SD can help clear all corpses files in your 2nd partitions and your phone memory...

Sent from my SK17i
 

2_i

Member
May 22, 2013
30
3
Have you ever try using the cleanup features in Link2SD to clean all those corpses file?
"Clean up 2nd partition" and "Clean up dalvik-cache" in Link2SD can help clear all corpses files in your 2nd partitions and your phone memory...

Yes, I use these periodically, but these don't touch the left-over files in /data. In fact, I looked around and could not find any other app around with that corpse function.
 

iPusak Gaoq™

Senior Member
Mar 21, 2012
2,163
1,172
Kuching
OK, it seems that only the corpse in /data/data matters for the success of reinstallation. That certainly speeds up the manual process :good:

did you try "system cleaner" on SD Maid? it might remove those corpses file or go to your devices settings/apps and look at the download list....any apps if gray out (without any icon) can be remove by uninstalling....
"insufficient storage" usually appears when you install lucky patcher (patch to core.jar)...if you do install lucky patcher then use lucky patcher to remove all odex file and clear dalvik cache....
 
  • Like
Reactions: 2_i

2_i

Member
May 22, 2013
30
3
did you try "system cleaner" on SD Maid? it might remove those corpses file or go to your devices settings/apps and look at the download list....any apps if gray out (without any icon) can be remove by uninstalling....
"insufficient storage" usually appears when you install lucky patcher (patch to core.jar)...if you do install lucky patcher then use lucky patcher to remove all odex file and clear dalvik cache....

I tried SD Maid's System Cleaner before, but it did not help - it did not remove the corpses. In settings/apps I see the grayed out corpses and if I remove an app there it can be thereafter reinstalled so while it is still a bit tedious, it can be done within a finite time. Thanks!

I did indeed use the Lucky Patcher. At this point it was gone together with most other apps but I brought it back to life. When removing odex files, it would work on that for a while and then cause a reboot. I repeated this a couple of times and cleaned the dalvik cache but all that on its own did not help - the apps in question could not be reinstalled. However, the overall issue maybe indeed tied to the Lucky Patcher. Since it keeps its hands off some apps, that would explain why some of them could be reinstalled without problems.

For the record, I also tried Clean Master, but it just ended removing useful files - no problem there in the end as I have a backup.
 

Dark3n

Recognized Developer
Sep 14, 2007
2,567
5,175
twitter.com
The SystemCleaner can't find corpses because it has no notion of "installed/uninstalled" or even apps, it doesn't know what an app is, it just knows files and folders, it scans everything and matches against static filters, meaning unaffected by your device setup.

You could also see if the apps are listed as "NOAPK" in AppControl, if so, you could search for "NOAPK" or "NO_APK" (not sure), which will filter only for that, then you can select all entries and do a mass uninstall/delete.

But the correct tool for this job would be the CorpseFinder.
 

KOLANICH

Senior Member
Dec 30, 2009
100
14
DeepTown
Dark3n, would you like setup an fdroid repo on your servers? I know that fdroid is mostly for opensource, but as I understand nothing limits you to use it for proprietary sw.
 

KOLANICH

Senior Member
Dec 30, 2009
100
14
DeepTown
@Dark3n, F-Droid is not just a market. You can setup your own market. You just need a folder in your site, an apk, an xml file with metadata and .htaccess file to open the xml as default file in that folder.
After you have setup it, you publish an url, users input this url into fdroid client and your app appears in the client.
 
Last edited:
  • Like
Reactions: denyo81

Mike B.

Recognized Contributor
Nov 17, 2011
3,140
3,173
/
Out of curiosity, what would be the advantage for the users and/or Dark3n of going through F-Droid instead of or in addition to the current options (Play Store + Dark3n server)? It seems like a lot of work and maintenance for no clear gains. I have never found a reason to grab the F-Droid client and I see a number of F-Droid users getting confused because of signature conflicts on various apps but perhaps I'm missing an obvious benefit?
 
Last edited:

gbueno6

Senior Member
Nov 20, 2011
405
140
Barcelona
Hi fellas!
I've got an small question for @Dark3n: I have toggleled every autostart option of Google Play Services (just like I did in kitkat) in my moto g GPE 5.0.1 but I see, every time I exit and re-enter the app that 4 of them are still "on" while the other options are "off". Is there anything wrong? (Screenshots attached)
Have a great day and thank you for this great app!
A proud owner of SDMaid Pro
 

Attachments

  • Screenshot_2014-12-23-20-00-07.png
    Screenshot_2014-12-23-20-00-07.png
    178.4 KB · Views: 378
  • Screenshot_2014-12-23-20-00-14.png
    Screenshot_2014-12-23-20-00-14.png
    179.3 KB · Views: 368
  • Like
Reactions: Dark3n

Dark3n

Recognized Developer
Sep 14, 2007
2,567
5,175
twitter.com
That's a pretty sneaky bug you caught there, that could have gone ages without being detected, thanks!

The wrong component (one that is not listed) is getting disabled, which is why you don't see a change.

To prevent unknown sideeffects, please toggle each entry that shows this behavior, once to the OFF state, then to the ON state again.
This way the component we wrongly disabled, will be enabled again.

For the curious, the details:
It only happens on Lollipop and only affects entries that end with "$[Component]"

This is what we want to do:
Code:
pm disable com.google.android.gms/com.google.android.gms.config.ConfigFetchService$PackageReceiver
It goes [app package name]/[full package path to outer class]$[Inner class]
The "$" means the broadcast receiver that starts on boot, is an inner class.

If you put that into the shell as root, we get this:
Code:
Component {com.google.android.gms/com.google.android.gms.config.ConfigFetchService} new state: disabled
Notice the last missing $PackageReceiver, that's because $anything is interpreted as a variable by the shell which then tries to expand this variable. Because there is no variable name defined, it is replaced with "nothing".

But that should not be an issue because all commands in SD Maid that use external input, such as packagenames, app names, filenames etc. are sanitized by escaping any special characters.
So the command we run actually looks more like this:
Code:
pm disable com\.google\.android\.gms\/com\.google\.android\.gms\.config\.ConfigFetchService\$PackageReceiver
And this works on <5.0, because we are directly executing that.

On 5.0 though we are using context switching, so the command looks similar to this:
Code:
su --context u:r:system_app:s0 -c "pm enable com\.google\.android\.gms\/com\.google\.android\.gms\.config\.ConfigFetchService\$PackageReceiver"
Which means we open another shell with a different context that then runs the command.
What i did not take into account is that each shell consumes the escape literals.
So we tell shell-A "do \$stuff", what shell-A then sees as final command is "do $stuff".
If we would pass this to a shell-B, that shells sees $stuff as variable and expands it to nothing, so what shell-B actually executes as command is just "do ".

What's the solution? We replace the outer double quotes that wrap the command with single quotes, the single quotes preserve the literal characters.
This means that the input the second shell receives still contains all escape characters like it was before <5.0 where we directly executed the command.
The final command then looks like this:
Code:
su --context u:r:system_app:s0 -c 'pm enable com\.google\.android\.gms\/com\.google\.android\.gms\.config\.ConfigFetchService\$PackageReceiver'

TLDR
' vs "
:(
 
Last edited:

Dark3n

Recognized Developer
Sep 14, 2007
2,567
5,175
twitter.com
I think i will push a beta for this as this can potentially break apps depending on what the component is that gets wrongly disabled.

Because SD Maid only shows Autostart components (i.e. on boot receivers), and the outter class most likely is not an on boot receiver, you can't actually fix this within SD Maid if this has caused problems.

I've looked at a few other apps that can toggle components and this one would do the trick, i.e. renable the wrongly disabled component, though it seems to be affected by a reverse issue as it doesn't actually show the nested receivers that caused the issue in the first place.

This leads me to the conclusion, that SD Maid should have an experimental/advanced section that shows all app components not just on boot receivers.
 

gbueno6

Senior Member
Nov 20, 2011
405
140
Barcelona
That's a pretty sneaky bug you caught there, that could have gone ages without being detected, thanks!

The wrong component (one that is not listed) is getting disabled, which is why you don't see a change.

To prevent unknown sideeffects, please toggle each entry that shows this behavior, once to the OFF state, then to the ON state again.
This way the component we wrongly disabled, will be enabled again.

For the curious, the details:
It only happens on Lollipop and only affects entries that end with "$[Component]"

This is what we want to do:

It goes [app package name]/[full package path to outer class]$[Inner class]
The "$" means the broadcast receiver that starts on boot, is an inner class.

If you put that into the shell as root, we get this:

Notice the last missing $PackageReceiver, that's because $anything is interpreted as a variable by the shell which then tries to expand this variable. Because there is no variable name defined, it is replaced with "nothing".

But that should not be an issue because all commands in SD Maid that use external input, such as packagenames, app names, filenames etc. are sanitized by escaping any special characters.
So the command we run actually looks more like this:

And this works on <5.0, because we are directly executing that.

On 5.0 though we are using context switching, so the command looks similar to this:

Which means we open another shell with a different context that then runs the command.
What i did not take into account is that each shell consumes the escape literals.
So we tell shell-A "do \$stuff", what shell-A then sees as final command is "do $stuff".
If we would pass this to a shell-B, that shells sees $stuff as variable and expands it to nothing, so what shell-B actually executes as command is just "do ".

What's the solution? We replace the outer double quotes that wrap the command with single quotes, the single quotes preserve the literal characters.
This means that the input the second shell receives still contains all escape characters like it was before <5.0 where we directly executed the command.
The final command then looks like this:


TLDR
' vs "
:(

I'm more than happy to help! I've been playing with it since kitkat but, considering I had a severe battery drain caused by Play Services, I decided to disable every autostart (just like I do with almost every app) option and then I saw that problem.

I wish I could help you more! If I can do anything for you, just tell me and I will for sure!

Have a nice day!

PD: I did what you asked me to avoid potential issues and it worked! (I couldn't check for updates, for example). Please, let me know if I can help.
 
Last edited:

Mike B.

Recognized Contributor
Nov 17, 2011
3,140
3,173
/
...
What's the solution? We replace the outer double quotes that wrap the command with single quotes, the single quotes preserve the literal characters.

Oh I enjoyed that confession! Those of us who have been dealing with Unix and its descendants since way before POSIX have all at one time or another weak quoted (") when we should have strong quoted ('). I suspect that countless debugging hours have been spent tracking down that nasty little difference over the years. Don't fret, you're in good company! ;) ;)
 

Top Liked Posts

  • There are no posts matching your filters.
  • 2
    It has been 5 months since I updated my device to Android 11 and App Cleaner is not working and I tried my best and it didn't help. How many times do I send the bug log and reports to githup and there is no solution @Dark3n @Oswald Boelcke
    I apologise I can't assist. I'm running LOS 16 and will never ever go beyond.
    1
    I've just download v5.1.1 Beta from Google PS beta programme and installed it. Changelog:
    And the last production version v5.1.0 is as of 2021-03-01; I wouldn't call that to be discontinued.

    The major functionalities are for free.
    It has been 5 months since I updated my device to Android 11 and App Cleaner is not working and I tried my best and it didn't help. How many times do I send the bug log and reports to githup and there is no solution @Dark3n @Oswald Boelcke
    1
    I hope you will inform them of our complaints about our feedback
    Negative. What makes you to assume so? I'm just a satisfied user of SD Maid.
    1
    Hi, what should I set to clear the cache of all apps?
    I bought the PRO version of the app, but the caches of all the apps do not appear, and the ones that are displayed are smaller than the one that shows the details of the app itself.
    For example: on Youtube Vanced, in SdMaid it tells me that I can clean 14mb, in the app manager it tells me that the cache is 53mb.
    This on Galaxy S7 Edge with android 8 and without root.
    1
    From the app's you've mentioned above I'm only using Signal and the indications nearly correspond.
    I apologise that I didn't switch system language this time.
    The ones I mentioned are just some of the ones that SDmaid just doesn't detect when scanning. In reality there are very few that it detects in 'cleaning applications' and as I said in the previous message in none of those displayed the cache is the actual one.
    Signal doesn't appear in SDmaid, yet through the app manager I know I can delete 13mb of cache.
    I do not understand why :(
  • 515
    Description:
    Ive started working on a tool to clean up after apps (inspired by this thread).

    Nobody is perfect and android neither.
    Apps you have already deleted sometimes leave data behind.
    The system constantly creates logs, crash reports and debug files you don't really need.
    The market leaves temporary files of downloaded apps behind.
    Lets not go on here...

    SD Maid is a universal file tool that helps you keep your device neat and tidy.


    Features
    • Storage overview
    • Root explorer & search
    • Find & remove left over files from uninstalled apps
    • Freeze, Uninstall, Export or kill apps
    • Toggle autstart entries
    • Find & delete common unnecessary files
    • Clean applications by removing various cache files without loosing any data
    • Find & delete duplicate files
    • Overview over biggest files/directories on your device
    • Optimize your SQL databases
    • Show last modified files
    • and more...

    Installation instructions
    * Install this like any other app.
    * If your device is rooted, you will be asked if you want to grant SD Maid root permission upon its first launch
    * It is up to you what you choose, in each case SD Maid will make best possible use of its features.
    * Have the SuperUser application remember this choice, repeated prompts can cause unexpected behavior

    Download links
    SD Maid can either be downloaded from Google Play or direct from my server, appropriate links are collected here. The unlocker for SD Maid Pro can currently only be obtained through Google Play. I really recommend to only use these two sources. If you can't verify that the apk files are original you may put your device at risk.

    Changelog
    A detailed changelog can be found on Github and most update release posts also contain some explanation of what's new.

    Bugs / Requests
    Please use the issue tracker for any kind of bug reports or feature requests for both SD Maid stable & beta. I'll also discuss issues else where but usually a lot of helpful information gets lost if it's not organised, so I prefer the issue tracker :).

    Beta testing
    If you are interested in beta versions and want to help test new features you can find the appropriate links here There is also some discussion in SD Maids G+ community.

    Translations
    If you want to help translate look here: SD Maid @ crowdin.com
    Thank you!

    Help document
    See sdmaid.darken.eu/help

    Privacy
    See sdmaid.darken.eu/privacy

    XDA:DevDB Information
    SD Maid - System Cleaning Tool, App for the Android General

    Contributors
    Dark3n

    Version Information
    Status: Stable
    Current Stable Version: -
    Stable Release Date: 1969-12-31

    Created 2014-06-02
    Last Updated 2017-08-15
    77
    How to help me troubleshoot your issues

    Please see here for instructions on how to best report bugs:
    https://github.com/d4rken/sdmaid-public/wiki/Reporting-a-bug
    31
    Weekend beta v4.3.0 brings a truckload of core improvements.

    Here are the links if you just want to dive in:

    http://sdmaid.darken.eu/download
    http://sdmaid.darken.eu/changelog
    http://sdmaid.darken.eu/issues

    Some notes:
    • Tools will now display 'allocated size' instead of 'apparent size'. Allocated size is the size an item actually takes up on your storage (i.e. a 1 byte file actually uses 4096 byte on storage). Apparent size is a files length, how large the file itself tells us it is (i.e. a file may say it is 8096 byte long, but actually only takes up 4096 byte). A big source for discrepancy between sizes are caused by "sparse-files", these are files that contains a lot of 0000000. If the file system supports it, a sparse file of length 1GB could be saved on a 1MB storage. I think we are mainly interested in size on our storage (allocated size), not how much data something uses when sending it over network (apparent size). This will be most noticeable with the AppCleaner (which will show larger amounts because of many small files) and the SystemCleaner (which will show smaller ammounts, due to thumbnails often being sparse files). The Explorer will show both sizes if they differ, format 'allocated size (apparent size)'.
    • SD Maid now ships with toybox instead of busybox. Toybox is smaller and it fixes an issue I had with busybox. It's debatable whether toybox is faster, but looking it's code it is at least a lot leaner. Toybox is also something that all future Android versions will ship which gives us a lot of room for good compatibility on future Android ROMs (e.g. if we have to use the system toybox binary instead of SD Maids own). Busybox was not installed in most ROMs. Why not both? SD Maid is now compatible with both toybox and busybox, and will fallback to a busybox on your system if that is what it takes to run without issue on your device. Maxium compatibility :)!
    • If you have root SD Maid will no longer ask for secondary storage permission. There were too many problems with people on chinese (rooted) ROMs that did not manage to grant the permission. You can still grant it manually via advanced settings. In theory, i thought performance should be better with secondary storage permissions through the storage access framework, but I couldn't actually measure it. ¯\_(ツ)_/¯
    • You will probably never see the 'BUSYBOX error' page again because this only happens with root and SD Maid will now give up root if it would otherwise mean showing the error and giving up.
    • AppControl doesn't load app sizes by default anymore and is thus much faster. Also has improved filtering options (right side drawer).
    • The tool "Biggest" is now called "Storage analyzer" and has been completely rewritten.
    • Explorer now supports unpacking any type of 'zip' files (.zip, .apk etc) (without root for now).
    • The LastModified tool is gone (bye!), and the Searcher now has options for minimum and maximum age (hi!).
    • There is a new (better?) clutter reporter tool which uses GitHub. Let me know how it works and what I can improve, clutter reports is what gets us closer and closer to a 100% detection rate.
    • Besides dozen of bugfixes, there have also been multiple core changes that attempt to improve performance of SD Maid by reusing objects and making data lookups faster.
    • Not that you can see it, but many of the changes made it also a lot easier for me to better write unit tests for modules of SD Maid :cool:

    All unit tests, I currently have, passed, but due to the amount of changes and changes to the core structure, I expect a few bugs, so this is a beta only release. Catching them and adding more tests is what a beta is for though, right :)? (give the issue tracker some love!)
    28
    I made some good progress on the weekend so SD Maid v4.10.0 is here.

    This update is dedicated to the duplicates tool and a beta-only release.

    Let's start with the biggest change: The previous autoselection dialog is gone and SD Maid now performs an automatic selection based on a set of criterias you can adjust. So now you press the autoselection button and it just happens.

    The previous dialog allowed you to select one and only one type of criterion to select files by. There isn't really a right and a wrong way to do that, but based on many mails I received there is certainly a popular preference. I've put these preferences as "defaults" into an extra screen (check the option menu) which you can adjust.

    Currently we have the following defaults:

    - Priority 1: Media Provider, try to keep files that are indexed by the Media Storage app and try to delete those that are not indexed. The most common type of support mail I get is people complaining that SD Maid deleted their images, because they only look in the gallery app, they think SD Maid deleted them all, but the remaining copy is just in a space that is not indexed.

    - Priority 2: Storage location, assuming the user has an extra sdcard, try to keep the files on the sdcard and delete the files in internal storage. The assumption is that the user always wants to free space on internal storage.

    - Priority 3: Path nesting, we prefer to keep shallowly nested files as those are easier to find and access.

    - Priority 4: Modification date, we prefer older files because we assume that the newer file is an unwanted copy. E.g. older picture comes from the camera app, while the newer picture is a copy from an instant messenger app.

    I'm looking for feedback whether you think this is the optimal default setup.

    You can also find these details in the wiki: https://github.com/d4rken/sdmaid-public/wiki/Duplicates

    The second new feature is "pruning media storage references". It's a not too rare occurence that apps show empty/grey thumbnails for image files that have been deleted, because the media storage database still contains a reference. SD Maid will now check the database for references to images deleted by the duplicates tool and then delete those references.

    There have also been various changes to the user interface. I'm slowly changing some of SD Maids core UI architecture to allow for a more responsive UI interaction. The duplicates tool is just the first tool getting this treatment. The UI should lag less on low end devices and it should also no longer freeze when running selection on thousands of files. Play a bit around with the autoselection, using single click/long click on items and let me know how it goes.

    I've also added a box that shows the task result prominently. Task results were previously rather hidden away (navdrawer, startpage, notifications) and you couldn't easily view them in the actual tool they belong too, which is a bit backwards, isn't it? I plan to add this "task result box" to other tools too. With more space to display infos we can then also make the result messages a bit more detailed.

    Some texts such as the selection criterion explanations are not translatable yet as I'm looking for your feedback on the wording. Is it clear what each option? Please let me know.

    Oh and the duplicates settings have been moved into their own file and gotten a "reset to default" option, once the other tools get this we can finally have a nice and clean backup option for settings :).

    There have been a few other changes, but I think this is enough wall of text. Check the changelog ;).

    http://sdmaid.darken.eu/changelog
    http://sdmaid.darken.eu/issues
    http://sdmaid.darken.eu/download
    28
    v4.10.1 is out.

    I've fixed reported crashes (thx @IronTechmonkey ) and spend time rewriting quick access and other progress related feedback to improve performance. In some case there were multiple dozen callbacks per second which just wasted UI resources. I've limited this to ~70ms update intervals which is about the lower edge of delay at which humans can notice motion.

    Using ConstraintLayout I've also made several UI hierarchies flatter which should also improve performance when redrawing elements to display progress updates.

    Now that the duplicate criteria screen no longer crashes, I'm looking for some feedback on the texts/wording of each option before they are made translateable.

    http://sdmaid.darken.eu/changelog
    http://sdmaid.darken.eu/issues
    http://sdmaid.darken.eu/download