Attend XDA's Second Annual Developer Conference, XDA:DevCon 2014!
5,739,962 Members 40,257 Now Online
XDA Developers Android and Mobile Development Forum

[falcon][CWM Advanced Edition] PhilZ Touch

Tip us?
 
droider007
Old
#511  
droider007's Avatar
Senior Member
Thanks Meter 204
Posts: 661
Join Date: Mar 2012
Location: kurukshetra
Hey guys....Mounting usb storage works...??
For me its not working...

Sent from my Moto G using XDA Free mobile app
SAYING THANKS = CLICKING
 
hachamacha
Old
#512  
Senior Member
Thanks Meter 279
Posts: 546
Join Date: Jun 2010
Quote:
Originally Posted by zarac View Post
A few questions (they might be silly): Do I execute this command in recovery or while booted? I tried it while booted and I got
Code:
Select Code
/system/bin/sh: tar: not found
Do I need busybox or something like that?

I typed the command as follows:
Code:
Select Code
tar -cpvzf /storage/emulated/0/test/data.tar.gz /data --exclude=data/data/com.google.android.music/files/* >> /storage/emulated/0/test/log.txt
Is that the correct syntax? Not sure about the '$' part from your post. Also, is the data directory just '/data' or '/data/data'?

I can test it in recovery as well, but only later tonight (7PM here now) because I'm expecting some rather important calls. I really shouldn't power-off the phone now, even for 10 minutes.

Thanks
It didn't occur to me that you might not have tar which (yes) is in busybox, although in newer recovery versions is built from {recovery}/libtar, so you would have to install busybox. It doesn't matter so much what it does when it's booted until we know what it does from recovery so wait until you've got time (tonight, whenever) and check it out. You'll have tar in recovery (might be /sys/xbin/tar or "/sys/xbin/busybox tar").

For you I'm most interested in the files that aren't reachable during the /data backup, so /data means "/data" since your unreachable files/dirs are under /data/dalvik*, etc. Forget the $ sign from my post. I shouldn't have used it. All the ${ or { sections are just meant to be real data substituted by you since you'll know the right places. I assume you're thinking about /data/data since the --exclude clause uses it, but we're just trying to duplicate what happens in the real recovery which excludes music files under /data/data.

I'll check back tonight or tomorrow.

---------- Post added at 10:28 AM ---------- Previous post was at 10:27 AM ----------

Quote:
Originally Posted by droider007 View Post
Hey guys....Mounting usb storage works...??
For me its not working...

Sent from my Moto G using XDA Free mobile app
I've never used it, don't have any gear for it, or a Moto G for that matter, but it is clear that @zarac has used it without any particular problem, so maybe he can help out here.
The Following 2 Users Say Thank You to hachamacha For This Useful Post: [ Click to Expand ]
 
zarac
Old
#513  
zarac's Avatar
Senior Member
Thanks Meter 50
Posts: 101
Join Date: Jan 2009
Location: Zagreb
Quote:
Originally Posted by droider007 View Post
Hey guys....Mounting usb storage works...??
For me its not working...

Sent from my Moto G using XDA Free mobile app
For me it is mounted automatically if I have the OTG drive plugged in when booting into recovery, I haven't tried plugging it afterwards. In backup options you should see 'Backup to /usbdisk' if it is mounted.
Moto G (falcon), XT1032 - Global GSM
rom and kernel: stock 4.4.4 (210.12.40.falcon_umts.Retail.en.GB) - rooted

HTC Desire (bravo), SLCD, PVT4, MTD erasesize 20000
rom and kernel: Evervolv Kitkat alpha (Android 4.4.4) built by LaidbackNikez

Asus Nexus 7 (grouper), 32GB
rom: Carbon Rom all-F2FS (Android 4.4.4) built by USBhost
kernel: M-Kernel a69 (f2fs/ext4) built by Metallice
 
hachamacha
Old
#514  
Senior Member
Thanks Meter 279
Posts: 546
Join Date: Jun 2010
@zarac:

As far as tar being a part of busybox, normally it is. Maybe a few (or more) months back, Philz Touch rewrote the tar code and put it into a busybox-like collection of tools called 'recovery' (./root/sbin/recovery), but I think there is a flag that allows switching between the tar built into busybox and the tar built into recovery, so the safest way to use what Philz Touch is using is to just use the symbolic link "tar" rather than "busybox tar" or "recovery tar". I hope that makes sense.

In fact, if both of them and the flag for switching still exist in this latest revision, we could also compare both of their outputs by manually switching them in the command, but more of that later. I just wanted to mention that the best course from inside recovery is to just use the command "tar" alone from adb or term emulator and let the symbolic link take care of which one it's running. I'm 99% sure it will run the recently added "recovery-built" one.

I'll also edit the prior post I put up asking you to do the tar command manually.
The Following User Says Thank You to hachamacha For This Useful Post: [ Click to Expand ]
 
zarac
Old
(Last edited by zarac; Yesterday at 07:58 PM.)
#515  
zarac's Avatar
Senior Member
Thanks Meter 50
Posts: 101
Join Date: Jan 2009
Location: Zagreb
Quote:
Originally Posted by hachamacha View Post
It didn't occur to me that you might not have tar which (yes) is in busybox, although in newer recovery versions is built from {recovery}/libtar, so you would have to install busybox. It doesn't matter so much what it does when it's booted until we know what it does from recovery so wait until you've got time (tonight, whenever) and check it out. You'll have tar in recovery (might be /sys/xbin/tar or "/sys/xbin/busybox tar").

For you I'm most interested in the files that aren't reachable during the /data backup, so /data means "/data" since your unreachable files/dirs are under /data/dalvik*, etc. Forget the $ sign from my post. I shouldn't have used it. All the ${ or { sections are just meant to be real data substituted by you since you'll know the right places. I assume you're thinking about /data/data since the --exclude clause uses it, but we're just trying to duplicate what happens in the real recovery which excludes music files under /data/data.

I'll check back tonight or tomorrow.[COLOR="Silver"]
OK, I got my calls and rebooted to recovery immediately.

I issued the following command
Code:
Select Code
tar -cpvzf /sdcard/Download/data.tar.gz /data --exclude=data/data/com.google.android.music/files/* >> /sdcard/Download/log.txt
After about 5 minutes it stopped with the following line (error?)
Code:
Select Code
tar_append_tree("/data", "/data"): No such file or directory
The resulting data.tar.gz is approx. 800MB.

After inspecting the command, I noticed that 'exclude' pointed to a relative path (data/data/com....) so i tried issuing the command again with an absolute path (/data/data/com...), like this:
Code:
Select Code
tar -cpvzf /sdcard/Download/data.tar.gz /data --exclude=/data/data/com.google.android.music/files/* >> /sdcard/Download/log2.txt
The result was pretty much the same (tar_append_tree error), apart from the fact that log2.txt was slightly smaller in size. I didn't get the chance to compare the data.tar.gz files from two runs as I deleted the first one before the second run, but they're both around 800MB, which should mean it is not complete - I expected a file of about 2.5GB 1.7GB (EDIT: my whole backup should be 2.5GB, not just 'data')

BTW, I used version 6.54.9 for these tests.
Attached Files
File Type: zip zarac_tar-logs.zip - [Click for QR Code] (170.1 KB, 2 views)
Moto G (falcon), XT1032 - Global GSM
rom and kernel: stock 4.4.4 (210.12.40.falcon_umts.Retail.en.GB) - rooted

HTC Desire (bravo), SLCD, PVT4, MTD erasesize 20000
rom and kernel: Evervolv Kitkat alpha (Android 4.4.4) built by LaidbackNikez

Asus Nexus 7 (grouper), 32GB
rom: Carbon Rom all-F2FS (Android 4.4.4) built by USBhost
kernel: M-Kernel a69 (f2fs/ext4) built by Metallice
The Following 2 Users Say Thank You to zarac For This Useful Post: [ Click to Expand ]
 
Phil3759
Old
#516  
Phil3759's Avatar
Recognized Developer - OP
Thanks Meter 30324
Posts: 9,162
Join Date: May 2012

 
DONATE TO ME
Quote:
Originally Posted by zarac View Post
OK, I got my calls and rebooted to recovery immediately.

I issued the following command
Code:
Select Code
tar -cpvzf /sdcard/Download/data.tar.gz /data --exclude=data/data/com.google.android.music/files/* >> /sdcard/Download/log.txt
After about 5 minutes it stopped with the following line (error?)
Code:
Select Code
tar_append_tree("/data", "/data"): No such file or directory
The resulting data.tar.gz is approx. 800MB.

After inspecting the command, I noticed that 'exclude' pointed to a relative path (data/data/com....) so i tried issuing the command again with an absolute path (/data/data/com...), like this:
Code:
Select Code
tar -cpvzf /sdcard/Download/data.tar.gz /data --exclude=/data/data/com.google.android.music/files/* >> /sdcard/Download/log2.txt
The result was pretty much the same (tar_append_tree error), apart from the fact that log2.txt was slightly smaller in size. I didn't get the chance to compare the data.tar.gz files from two runs as I deleted the first one before the second run, but they're both around 800MB, which should mean it is not complete - I expected a file of about 2.5GB 1.7GB (EDIT: my whole backup should be 2.5GB, not just 'data')

BTW, I used version 6.54.9 for these tests.
Thanks for your patience

Can you try this, while booted in recovery and from an adb console

Code:
Select Code
adb shell
mount /data
cd / ; touch /sdcard/data.tar.gz ; set -o pipefail ; tar -csv --exclude=data/data/com.google.android.music/files/* --exclude 'media' data | pigz -c -3 | split -a 1 -b 1000000000 /proc/self/fd/0 /sdcard/data.tar.gz. ; echo $?
if it fails, try busybox tar:
Code:
Select Code
adb shell
mount /data
cd / ; touch /sdcard/data.tar.gz ; set -o pipefail ; busybox tar -csv --exclude=data/data/com.google.android.music/files/* --exclude 'media' data | pigz -c -3 | split -a 1 -b 1000000000 /proc/self/fd/0 /sdcard/data.tar.gz. ; echo $?
>PhilZ Touch Recovery: Main thread and Download links
> Support IRC Channel: freenode - #PhilZ-Touch
>Root ICS/JB for all phones: link
>CWM6 Temporary: i9100, n7000
>Busybox Uninstaller: safely remove +600 files
>Safe Wipe Data/System/Cache/Dalvik: i9100/n7000
>CF-Root in Stock Recovery: dead project
The Following User Says Thank You to Phil3759 For This Useful Post: [ Click to Expand ]
 
hachamacha
Old
(Last edited by hachamacha; Yesterday at 08:49 PM.) Reason: hide to see what Philz's request brings forth
#517  
Senior Member
Thanks Meter 279
Posts: 546
Join Date: Jun 2010
I see now that I've written this that Phil's got a request for you so as not to confuse things I'm sticking some hide tags on mine. Ignore this post for now, (maybe forever) ;) Thanks.

 
Quote:
Originally Posted by zarac View Post
OK, I got my calls and rebooted to recovery immediately.

I issued the following command
Code:
Select Code
tar -cpvzf /sdcard/Download/data.tar.gz /data --exclude=data/data/com.google.android.music/files/* >> /sdcard/Download/log.txt
After about 5 minutes it stopped with the following line (error?)
Code:
Select Code
tar_append_tree("/data", "/data"): No such file or directory
The resulting data.tar.gz is approx. 800MB.

After inspecting the command, I noticed that 'exclude' pointed to a relative path (data/data/com....) so i tried issuing the command again with an absolute path (/data/data/com...), like this:
Code:
Select Code
tar -cpvzf /sdcard/Download/data.tar.gz /data --exclude=/data/data/com.google.android.music/files/* >> /sdcard/Download/log2.txt
The result was pretty much the same (tar_append_tree error), apart from the fact that log2.txt was slightly smaller in size. I didn't get the chance to compare the data.tar.gz files from two runs as I deleted the first one before the second run, but they're both around 800MB, which should mean it is not complete - I expected a file of about 2.5GB 1.7GB (EDIT: my whole backup should be 2.5GB, not just 'data')

BTW, I used version 6.54.9 for these tests.
Thanks for the work. From the two logs you supplied (which are identical) it appears that tar makes it through a large number (well, 800MB or so) of /data/* files including a lot of /data/data ones without trouble until it gets to , I think, this one: persist.atvc.log (True Loc: /data/property/persist.atvc.log) at which point it appears not to see it anymore. If it didn't exist in a directory somewhere, it wouldn't be in the backup list, so it makes me think that you may have a corrupt /data FS.

For me to be sure it's not just recovery's problem, I need to make sure of the true "root" folders names during recovery's run which I'll do on my device (a Moto X right now). My FS's are basically the same types and names as yours on the Moto G so it should be similar enough. I don't have the problems with the f2fs /data FS so maybe I can figure out whether we're looking at something that needs tweaking in say, extra.fstab or some build parameter.

One other thing that bugs me (but this is without yet reading the tar code) is why backup would be backing up /data/media/0 which must be huge, too large to be backed up, yet it is also among those folders being tried unsuccessfully. I guess I can only "see" that as a target for backups, not as a source unless a few "./" folders are being backed up to save settings. The reason I know backup is trying to do that is because of the original failure log you posted a few pages back. The manual tar command I asked you to do wouldn't have discriminated so backing up all of data wouldn't be a surprise there aside from whatever I put on the "--except=" clause.

Anyway, I'll check it further and get back to you.
The Following User Says Thank You to hachamacha For This Useful Post: [ Click to Expand ]
 
Phil3759
Old
#518  
Phil3759's Avatar
Recognized Developer - OP
Thanks Meter 30324
Posts: 9,162
Join Date: May 2012

 
DONATE TO ME
Quote:
Originally Posted by hachamacha View Post
I see now that I've written this that Phil's got a request for you so as not to confuse things I'm sticking some hide tags on mine. Ignore this post for now, (maybe forever) Thanks.

 


Thanks for the work. From the two logs you supplied (which are identical) it appears that tar makes it through a large number (well, 800MB or so) of /data/* files including a lot of /data/data ones without trouble until it gets to , I think, this one: persist.atvc.log (True Loc: /data/property/persist.atvc.log) at which point it appears not to see it anymore. If it didn't exist in a directory somewhere, it wouldn't be in the backup list, so it makes me think that you may have a corrupt /data FS.

For me to be sure it's not just recovery's problem, I need to make sure of the true "root" folders names during recovery's run which I'll do on my device (a Moto X right now). My FS's are basically the same types and names as yours on the Moto G so it should be similar enough. I don't have the problems with the f2fs /data FS so maybe I can figure out whether we're looking at something that needs tweaking in say, extra.fstab or some build parameter.

One other thing that bugs me (but this is without yet reading the tar code) is why backup would be backing up /data/media/0 which must be huge, too large to be backed up, yet it is also among those folders being tried unsuccessfully. I guess I can only "see" that as a target for backups, not as a source unless a few "./" folders are being backed up to save settings. The reason I know backup is trying to do that is because of the original failure log you posted a few pages back. The manual tar command I asked you to do wouldn't have discriminated so backing up all of data wouldn't be a surprise there aside from whatever I put on the "--except=" clause.

Anyway, I'll check it further and get back to you.
I also agree that this seems like a corrupted file system since it always fails on same files
Using busybox tar from my suggestion above should clear up things
>PhilZ Touch Recovery: Main thread and Download links
> Support IRC Channel: freenode - #PhilZ-Touch
>Root ICS/JB for all phones: link
>CWM6 Temporary: i9100, n7000
>Busybox Uninstaller: safely remove +600 files
>Safe Wipe Data/System/Cache/Dalvik: i9100/n7000
>CF-Root in Stock Recovery: dead project
 
zarac
Old
#519  
zarac's Avatar
Senior Member
Thanks Meter 50
Posts: 101
Join Date: Jan 2009
Location: Zagreb
OK, more weirdness.
Command 'mount /data' got me 'device or resource busy' error, and I guess it is OK since 'data' was already automatically mounted (I checked via GUI under 'mounts')
But both other commands immediately gave tar syntax error. No files were created.

Code:
Select Code
~ # mount /data
mount: mounting /dev/block/platform/msm_sdcc.1/by-name/userdata on /data failed: Device or resource busy

~ # cd / ; touch /sdcard/data.tar.gz ; set -o pipefail ; tar -csv --exclude=data/data/com.google.android.music/files/* --exclude 'media' data | pigz -c -3 | split -a 1 -b 1000000000 /proc/self/fd/0 /sdcard/data.tar.gz. ; echo $?

tar: invalid option -- s

~ # cd / ; touch /sdcard/data.tar.gz ; set -o pipefail ; busybox tar -csv --exclude=data/data/com.google.android.music/files/* --exclude 'media' data | pigz -c -3 | split -a 1 -b 1000000000 /proc/self/fd/0 /sdcard/data.tar.gz. ; echo $?

busybox: invalid option -- s
Quote:
Originally Posted by Phil3759 View Post
I also agree that this seems like a corrupted file system since it always fails on same files
Using busybox tar from my suggestion above should clear up things
But even if the fs is corrupted, how come it works fine with v6.26.6 and earlier?
Moto G (falcon), XT1032 - Global GSM
rom and kernel: stock 4.4.4 (210.12.40.falcon_umts.Retail.en.GB) - rooted

HTC Desire (bravo), SLCD, PVT4, MTD erasesize 20000
rom and kernel: Evervolv Kitkat alpha (Android 4.4.4) built by LaidbackNikez

Asus Nexus 7 (grouper), 32GB
rom: Carbon Rom all-F2FS (Android 4.4.4) built by USBhost
kernel: M-Kernel a69 (f2fs/ext4) built by Metallice
 
Phil3759
Old
#520  
Phil3759's Avatar
Recognized Developer - OP
Thanks Meter 30324
Posts: 9,162
Join Date: May 2012

 
DONATE TO ME
Quote:
Originally Posted by zarac View Post
OK, more weirdness.
Command 'mount /data' got me 'device or resource busy' error, and I guess it is OK since 'data' was already automatically mounted (I checked via GUI under 'mounts')
But both other commands immediately gave tar syntax error. No files were created.

Code:
Select Code
~ # mount /data
mount: mounting /dev/block/platform/msm_sdcc.1/by-name/userdata on /data failed: Device or resource busy

~ # cd / ; touch /sdcard/data.tar.gz ; set -o pipefail ; tar -csv --exclude=data/data/com.google.android.music/files/* --exclude 'media' data | pigz -c -3 | split -a 1 -b 1000000000 /proc/self/fd/0 /sdcard/data.tar.gz. ; echo $?

tar: invalid option -- s

~ # cd / ; touch /sdcard/data.tar.gz ; set -o pipefail ; busybox tar -csv --exclude=data/data/com.google.android.music/files/* --exclude 'media' data | pigz -c -3 | split -a 1 -b 1000000000 /proc/self/fd/0 /sdcard/data.tar.gz. ; echo $?

busybox: invalid option -- s


But even if the fs is corrupted, how come it works fine with v6.26.6 and earlier?
sorry, that was the old command:

proper: (remove previous /sdcard/data.tar* files)
Code:
Select Code
adb shell
mount /data
cd / ; touch /sdcard/data.tar.gz ; set -o pipefail ; tar -cpv --exclude=data/data/com.google.android.music/files/* --exclude=data/media data | pigz -c -3 | split -a 1 -b 1000000000 /proc/self/fd/0 /sdcard/data.tar.gz. ; echo $?
busybox tar:
Code:
Select Code
adb shell
mount /data
cd / ; touch /sdcard/data.tar.gz ; set -o pipefail ; busybox tar -cpv --exclude=data/data/com.google.android.music/files/* --exclude=data/media data | pigz -c -3 | split -a 1 -b 1000000000 /proc/self/fd/0 /sdcard/data.tar.gz. ; echo $?
>PhilZ Touch Recovery: Main thread and Download links
> Support IRC Channel: freenode - #PhilZ-Touch
>Root ICS/JB for all phones: link
>CWM6 Temporary: i9100, n7000
>Busybox Uninstaller: safely remove +600 files
>Safe Wipe Data/System/Cache/Dalvik: i9100/n7000
>CF-Root in Stock Recovery: dead project

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes