Question system backup

[WoKoschekk]

I edited the Termux output for tar.

 

[m0d hipp¥]

ok I installed termux and ran pkg update to update all modules.

 

[WoKoschekk]

do you understand the "internal storage - /sdcard"-thing and why to exclude it?

It should also be possible to run Termux via ADB. Give me a few minutes to figure it out how to do it.

 

[m0d hipp¥]

Yes I understand it. basically you can't backup to it while it's in use. so it needs to be excluded. So since these are all the same paths "/data/media = /storage/emulated/0 = /sdcard", would it be enough to just exclude "/media*" or would you need to exclude "/data/media*", and "/storage/emulated/0*", and "/sdcard*"

Thanks!

 

[WoKoschekk]

Yes I understand it. basically you can't backup to it while it's in use. so it needs to be excluded. So since these are all the same paths "/data/media = /storage/emulated/0 = /sdcard", would it be enough to just exclude "/media*" or would you need to exclude "/data/media*", and "/storage/emulated/0*", and "/sdcard*"

Thanks!

exclude just /media as in the given command.

To use Termux via ADB:

adb shell
su #Magisk prompts you to grant root access for shell
PATH=/data/data/com.termux/files/usr/bin
SHELL=/data/data/com.termux/files/usr/bin/sh

 

[WoKoschekk]

exclude just /media as in the given command.

To use Termux via ADB:

adb shell
su #Magisk prompts you to grant root access for shell
PATH=/data/data/com.termux/files/usr/bin
SHELL=/data/data/com.termux/files/usr/bin/sh

Open Termux on your phone and run 'pkg install tsu' to install 'Termux su'. After that and after 'SHELL=/data...' you can run 'tsu'. That then brings you in a clean way into the Termux shell.

 

[m0d hipp¥]

Open Termux on your phone and run 'pkg install tsu' to install 'Termux su'. After that and after 'SHELL=/data...' you can run 'tsu'. That then brings you in a clean way into the Termux shell.

I'll try these commands and see if I can get it. Basically I would use this command right?

tar -cv --exclude='media/*' --exclude='*-cache' --file=/data/media/0/data.ext4.win000 /data

 

[WoKoschekk]

I'll try these commands and see if I can get it. Basically I would use this command right?

tar -cv --exclude='media/*' --exclude='*-cache' --file=/data/media/0/data.ext4.win000 /data

why did you add --exclude='*-cache'? This would exclude /data/dalvik-cache since there's no other path with '-cache' on /data.

 

[m0d hipp¥]

I added the cache because I came across this post: https://github.com/dlenski/tetherback That dev made a comment that for twrp backup for the data mount point

tar -cz -p --exclude="media*" --exclude="*-cache"

So I thought to include that as well.

 

[WoKoschekk]

I added the cache because I came across this post: https://github.com/dlenski/tetherback That dev made a comment that for twrp backup for the data mount point


So I thought to include that as well.

This was meant to be for Nexus 5 and a directory called data-cache. I assume it's an app cache but it's not used anymore on modern Android devices. Instead of this the option you add won't save ~/dalvik-cache. In general that's not an issue and on first boot the system will recreate the files. But it takes a few minutes and you could skip the waiting time by not excluding it.

I noticed that the guy uses --exclude='media*' while I used --exclude='media/*'.
Forget them both!

Let's pretend your partition stores

/data/mediafile
/data/media/0

BTW: In a previous post I said that /data/media = /storage/emulated/0 what is indeed /data/media/0 = /storage/emulated/0. I forgot the folder '0' that is specific to the user. You are user_0 and every new one will be created ongoing as user_1, ..., user_n.

To get to the point, the following explains the different arguments for --excluding:

# excluding 'media/*' saves:
/data/
/data/media
/data/mediafile

=> /data/media/0 is excluded


# excluding 'media*' saves:
/data/

=> all folders and also ALL FILES beginning with 'media' are excluded what is NOT desired!


# excluding 'media' saves:
/data/
/data/mediafile

=> this is the best practice and I would
recommend you to use this argument.
the folder ~/media/0 gets created on first boot by
the system. So, you don't need to save
/data/media with your backup. but apart from
that this argument will save all files beginning
with 'media'.

f.y.i. I haven't tested yet to restore a backup like this! I was searching for a method to create a TWRP backup manually. The command is able to do it 1:1 and a hexdump of the first lines is identical. But it can only be used in running system. I have got no idea what will happen if it's being restored.

 

[m0d hipp¥]

Thanks for the explanation and clarifying everything. I had a chance to test this out and here are my results.

I created 2 small scripts

batch script

@echo off
cls

pushd "%~dp0"

adb shell su <test.txt

Pause

test.txt

cd "/data/data/com.termux/files/usr/bin"
tar -cv --exclude='media' --file=/data/media/0/data.ext4.win000 /data

It took less than 1 minute to run which is pretty fast. It saved the "data.ext4.win000" data file and it shows 20Gb, however the last line in the output of the batch file states "tar: had errors". Is there a way I can see what the errors are?

 

[WoKoschekk]

Thanks for the explanation and clarifying everything. I had a chance to test this out and here are my results.

I created 2 small scripts

batch script

@echo off
cls

pushd "%~dp0"

adb shell su <test.txt

Pause

test.txt

cd "/data/data/com.termux/files/usr/bin"
tar -cv --exclude='media' --file=/data/media/0/data.ext4.win000 /data

It took less than 1 minute to run which is pretty fast. It saved the "data.ext4.win000" data file and it shows 20Gb, however the last line in the output of the batch file states "tar: had errors". Is there a way I can see what the errors are?

did you grant root permissions to "shell"?

The line »cd "/data/data/com.termux/files/usr/bin"« is useless in your case.

 

[m0d hipp¥]

did you grant root permissions to "shell"?

The line »cd "/data/data/com.termux/files/usr/bin"« is useless in your case.

the su is in the batch script. I tested it out by the command "whoami" and it returned root.

As far as the cd command, I thought that the tar file in termux is different than the standard tar issued with Android, so I was under the impression I had to cd to the bin file and execute the tar commands from there. Am I mistaken?

 

[WoKoschekk]

Am I mistaken?

yes, you are in SHELL=/system/bin/sh and therefore /system/bin/tar will be used by the system. That's why I told you to change the shell variables PATH= and SHELL=.

Instead of changing the shell environment you can also execute the binary in a current path you are in with
./tar
or from outside that path with /data/data/com.termux/files/usr/bin/./tar
For me this method seems to be a bit tricky and could lead to confusion if you forget the prefixed "./" To get around this I told you to change the environment.

So, test.txt should look like this:

cd "/data/data/com.termux/files/usr/bin"
./tar -cv --exclude='media' --file=/data/media/0/data.ext4.win000 /data

add a prefixed ./ to tar and you needn't to change the environment. But remember that for some up coming changes in your script!

 

[m0d hipp¥]

Sorry I totally missed the part on how to set environmental variables. I just did that now. I'll give my script another go.

By the way, when I set the path and shell variables is that only within the current session or is it permanent now any time I use ADB?

EDIT:
So I think the variables are only within the current session.
Anyway After setting the paths I reran the command

tar -cv --exclude='media' --file=/data/media/0/data.ext4.win000 /data

I see everything processing with the verbose output.
This time it ran with no errors, but it shows it's only 50mb. That can't be right.

 

[m0d hipp¥]

There was an error with my syntax. I got it now. It shows 20Gb and no errors.

 

[WoKoschekk]

By the way, when I set the path and shell variables is that only within the current session or is it permanent now any time I use ADB?

These variables are only set for the current session like any other variables you declare in a script. You can call the value with echo $PATH or echo $SHELL or all values with set.

 

[WoKoschekk]

instead of changing the variables you can also execute

/data/data/com.termux/files/usr/bin/./tsu

tsu = Termux su-shell, pkg install tsu
then you are also using the Termux shell as root.

 

[WoKoschekk]

Is there a way I can see what the errors are?

Rename data.ext4.win000 to data.ext4.win000.tar and it's readable and can be opened with any archiver software like WinRAR or 7z on Android/Windows. That's the easiest way to check the archive for errors/missing files.

 

[m0d hipp¥]

I still seem to be getting mixed results. sometimes I get a 20Gb backup other times it's only 50mb. I don't get any errors, but when I open up the 50mb archive I see several directories missing even though it processed without any errors. for example the 50mb file these are the directories within root that it backed up.

anr
bootchart
misc
tombstones
unencrypted
vendor
vendor_ce
vendor_de