FORUMS

[TOOL] ADB-Overlay [1.15.46.1]

706 posts
Thanks Meter: 669
 
By k1ll3r8e, Senior Member on 13th April 2019, 10:08 PM
Post Reply Email Thread
29th January 2020, 11:47 AM |#11  
k1ll3r8e's Avatar
OP Senior Member
Flag Delmenhorst
Thanks Meter: 669
 
Donate to Me
More
Quote:
Originally Posted by skyrail01

Great job. we can navigate and copy files efficiently.
Only one thing is missing. Why not keep file timestamp for last access and modifie dates? or let us choose the option to keep it or not.

Heyho,
which Android version are you using and on which action (pull/push) does the timestamps get lost?
Normally i preserve the timestamps but on some Android versions the "timestamp" field in the ADB-Protocol is ignored.

Maybe i can set the timestamps on an other way, but therefore i need to know the version.

Regards,
Sebastian

PS:
Some code where i set the timestamps

Upload
Code:
 public void Close(DateTime fileCreationTime)
        {
            if (_ch?.IsConnected ?? false)
            {
                if (CanWrite)
                {
                    int o = 0, w;
                    byte[] d = new byte[8];
                    d.SetUInt32LE((UInt32)SyncCmd.DONE, 0);
                    d.SetUInt32LE(ToUnixTime(fileCreationTime), 4);
                    //...
                }
                _ch.Close();
            }
        }
Download
Code:
//...
                using (FileStream dfs = destination.Open(FileMode.Create, FileAccess.Write))
                {
                    using (AFileStream sfs = OpenRead())
                    {
                        sfs.CopyTo(dfs, DMAX, msg, cancellationToken);
                        ct = sfs.CreationTime;
                    }
                }
//...
            if (msg.State == TransferState.Done)
            {
                destination.CreationTime = ct;
                transferEventHandler?.Invoke(Client.ID, new FileTransferEventArgs(Client.ID, msg));
                return true;
            }
//...
The Following User Says Thank You to k1ll3r8e For This Useful Post: [ View ] Gift k1ll3r8e Ad-Free
30th January 2020, 01:18 AM |#13  
Junior Member
Flag paris
Thanks Meter: 5
 
More
Hey,

I am operating under lineageOS 16 on Galaxy S5 (klte), using adb 1.0.39 on windows 10 64-bits, either via wifi or usb connect.
I check file dates under Windows, in console mode with far manager (M: last write time, C: creation time, A: last access time).
I am using linux command ls -l under adb shell from Windows command line to list files pushed to Android.

When pushing files from Windows to Android, only M is preserved, with a 2 hours offset (ES file explorer tells it is modified date) Maybe daylight issue or bad timezone.
When pulling files from Android, only C is preserved
When I pull back to Windows, files pushed to Android, all original date is lost.

But I am pretty sure we can preserve last access, modified and creation date as I could see Adbexplorer (another project) pull files keeping M and A (adb pull -a).
Hope I give you enough details.


Quote:
Originally Posted by k1ll3r8e

Heyho,
which Android version are you using and on which action (pull/push) does the timestamps get lost?
Normally i preserve the timestamps but on some Android versions the "timestamp" field in the ADB-Protocol is ignored.

Maybe i can set the timestamps on an other way, but therefore i need to know the version.

Regards,
Sebastian

PS:
Some code where i set the timestamps


Upload
Code:
 public void Close(DateTime fileCreationTime)
        {
            if (_ch?.IsConnected ?? false)
            {
                if (CanWrite)
                {
                    int o = 0, w;
                    byte[] d = new byte[8];
                    d.SetUInt32LE((UInt32)SyncCmd.DONE, 0);
                    d.SetUInt32LE(ToUnixTime(fileCreationTime), 4);
                    //...
                }
                _ch.Close();
            }
        }
Download
Code:
//...
                using (FileStream dfs = destination.Open(FileMode.Create, FileAccess.Write))
                {
                    using (AFileStream sfs = OpenRead())
                    {
                        sfs.CopyTo(dfs, DMAX, msg, cancellationToken);
                        ct = sfs.CreationTime;
                    }
                }
//...
            if (msg.State == TransferState.Done)
            {
                destination.CreationTime = ct;
                transferEventHandler?.Invoke(Client.ID, new FileTransferEventArgs(Client.ID, msg));
                return true;
            }
//...

The Following User Says Thank You to skyrail01 For This Useful Post: [ View ] Gift skyrail01 Ad-Free
30th January 2020, 02:16 AM |#14  
k1ll3r8e's Avatar
OP Senior Member
Flag Delmenhorst
Thanks Meter: 669
 
Donate to Me
More
Quote:
Originally Posted by skyrail01

Hey,

I am operating under lineageOS 16 on Galaxy S5 (klte), using adb 1.0.39 on windows 10 64-bits, either via wifi or usb connect.
I check file dates under Windows, in console mode with far manager (M: last write time, C: creation time, A: last access time).
I am using linux command ls -l under adb shell from Windows command line to list files pushed to Android.

When pushing files from Windows to Android, only M is preserved, with a 2 hours offset (ES file explorer tells it is modified date) Maybe daylight issue or bad timezone.
When pulling files from Android, only C is preserved
When I pull back to Windows, files pushed to Android, all original date is lost.

But I am pretty sure we can preserve last access, modified and creation date as I could see Adbexplorer (another project) pull files keeping M and A (adb pull -a).
Hope I give you enough details.

Heyho,
THX for that info, i'll check the "adbd" source which LOS uses, maybe there is the prob. According to "adb pull -a" i'll also have a look at the adb source, because i think they do some "touch" stuff. Because the ADB-Protocol it self only supports creation/last modified time. In the source is a another "struct" for "STA2" (stat 2) command. But i have never seen a adb binary which supports this. (All binaries use the "STAT" command.)

EDIT 1:
Just checked the adb source again, seems they changed it a bit...
(This is the current STAT (v1) header)
Code:
# cmd name STAT or STA2                 // 4 bytes
st->st_mode = msg.stat_v1.mode;     // 4 bytes
st->st_size = msg.stat_v1.size;            // 4 bytes
st->st_ctime = msg.stat_v1.mtime;    // 4 bytes
st->st_mtime = msg.stat_v1.mtime;  // 4 bytes
My HTC-One M9, Sensation, VM-Ware (Kitkat - Nougat) none of them has a 20 byte header. ALL of them return a 16 byte header.
Code:
CMD       // 4 bytes
Mode     // 4 bytes
Size        // 4 bytes
CTime    // 4 bytes
So it seems the protocol is highly inconsistent... I'll check how much time gets lost by calling touch on each file, because, currently i open a sync channel via an smart-socket, transfer the file and read all data provided by the protocol header. This is really fast, but if i have to spin up each time another channel to get a shell for the touch, this will increase the time each copy needs. Because, a shell depending on the version (v1, v2), has a lot checks you need to perform, specially the shell_v2 because it's like a ssh protocol.

EDIT 2:
Just for completeness, i'm pretty sure your device also sends only a 16 byte header for the stat v1 command, because:
(This is a cut of my implemented "Stat(byte[] bytes)" class)
Code:
if (c == (uint)SyncCmd.STA2 && bytes.Length >= 72)
            {
                ID = SyncCmd.STA2;
                Error = bytes.GetUInt32LE(4);
                Dev = bytes.GetUInt64LE(8);
                INode = bytes.GetUInt64LE(16);
                Mode = bytes.GetUInt32LE(24);
                NLink = bytes.GetUInt32LE(28);
                UID = bytes.GetUInt32LE(32);
                GID = bytes.GetUInt32LE(36);
                Size = bytes.GetUInt64LE(40);
                ATime = bytes.GetInt64LE(48);
                MTime = bytes.GetInt64LE(56);
                CTime = bytes.GetInt64LE(64);
            }
            else if (c == (uint)SyncCmd.STAT && bytes.Length == 16)
            {
                ID = SyncCmd.STAT;
                Mode = bytes.GetUInt32LE(4);
                Size = bytes.GetUInt32LE(8);
                CTime = bytes.GetUInt32LE(12);
            }
As you can see, the both structures differ in there field order, if your device would send a 20 byte header, the test (bytes.Length == 16) wouldn't match. And you couldn't download a file.

EDIT 3:
After playing a while with "stat" and "touch" on some Android versions, i decided to rely on those functions. The next build should restore and write the correct timestamps.

EDIT 4:
Well, setting timestamps on the SD-Card doesn't work, also, setting timestamps on the "internal" SD-Card doesn't work either. Seems that it's related to the underlying FS. If i try to set the timestamp on an ext4 partition, it works. So it doesn't make sense to rewrite the push/pull functions because the return of touch for 99% of the time will be "touch: '/sdcard/TWRP/test.pdf': Operation not permitted". So it's simply a time waste during the copy procedure. BUT, i'll implement it at least for the download, so u'll get the correct timestamps on Windows.

EDIT 5:
Added a screenshot of the new "Properties Window" which also proves that the download timestamps are now correct.
(Btw. C doesn't mean created - it means changed)
Code:
stat /sdcard
  File: `/sdcard' -> `/storage/self/primary'
  Size: 21	 Blocks: 0	 IO Blocks: 512	symbolic link
Device: 1h/1d	 Inode: 4432	 Links: 1
Access: (777/lrwxrwxrwx)	Uid: (    0/    root)	Gid: (    0/    root)
Access: 1970-01-01 01:00:00.000000000
Modify: 1970-01-01 01:00:00.000000000
Change: 1970-01-01 01:00:00.360063176
Little update about the "creation time". After a bit research i found this post on Stack Exchange, there is explained that the so called "birth time" is stored since ext4 as part of the INode field. But it seems you need a special version of "stat" to read this info, it's called "statx". But so far i could see, Android doesn't support this feature, maybe some kernel does but it seems that it's more the rare case.

Regards,
Sebastian
Attached Thumbnails
Click image for larger version

Name:	timestamp.png
Views:	50
Size:	73.4 KB
ID:	4939125  
6th February 2020, 12:52 PM |#15  
squabbi's Avatar
Senior Member
Flag Sydney
Thanks Meter: 1,604
 
Donate to Me
More
Thanks for the update!

I was wondering what could be causing a small delay in copying files over, once the progress finishes, the transfer dialog remains frozen for a 1-2 seconds. I'm copying over a photo, so around 2MB.
6th February 2020, 03:30 PM |#16  
k1ll3r8e's Avatar
OP Senior Member
Flag Delmenhorst
Thanks Meter: 669
 
Donate to Me
More
Quote:
Originally Posted by squabbi

Thanks for the update!

I was wondering what could be causing a small delay in copying files over, once the progress finishes, the transfer dialog remains frozen for a 1-2 seconds. I'm copying over a photo, so around 2MB.

Heyho ,
yup i still working on it. It seems that the "Task.Delay()" isn't performed inside the Task itself, instead it's executed on the main thread. The reason for the freeze is, that i try to show the last result, otherwise the copy window would close immediately after the last element and you couldn't see if it's transferred or not. So i keep it like windows, wait 1,5-2,5 sec before closing the window.

The delay you might feel extremely is the "Media-Scanner". It runs on the end of each copy/delete action, the reason for this is, that files pushed/copied by adb are not reported to the MTP interface. So you couldn't see those files if you connect your device after such an adb copy via MTP. Also files/folders deleted by adb are also not reported to the MTP interface. So you will see those elements via MTP if you don't perform manually the "Media-Scanner" lookup. - But if you work only with adb for example, you could turn the "Media-Scanner" off. All device based explorers and of course adb will see those files/folders at any times. (But if you need those files/folders for exchange you should keep the "Media-Scanner" on )

EDIT:
This is executed for every file which is copied/pushed/deleted.
Code:
_adb.Device.Manager.ActivityManager.Broadcast("android.intent.action.MEDIA_SCANNER_SCAN_FILE", "-a", string.Concat("-d 'file://", remotePath, "' > /dev/null"));
EDIT 2 (a bit off-topic):
You could also simply use the "Media-Scanner" with my AndroidCtrl.dll. The calls are:
Code:
//"_adb" is a instance of "ADBClient"
_adb.Device.IO.MediaScanner.CreateScanList(string remoteDirectory, MediaScanMode mode);
_adb.Device.IO.MediaScanner.CreateScanList(string remoteDirectory, MediaScanMode mode, CancellationToken cancellationToken);
_adb.Device.IO.MediaScanner.CreateScanList(string remoteDirectory, IList<string> files, MediaScanMode mode, CancellationToken cancellationToken);
_adb.Device.IO.MediaScanner.ScanDirectory(string remotePath, MediaScanMode mode);
_adb.Device.IO.MediaScanner.ScanDirectory(string remotePath, MediaScanMode mode, CancellationToken cancellationToken);
_adb.Device.IO.MediaScanner.ScanDirectory(IEnumerable<string> remotePaths, MediaScanMode mode);
_adb.Device.IO.MediaScanner.ScanDirectory(IEnumerable<string> remotePaths, MediaScanMode mode, CancellationToken cancellationToken);
_adb.Device.IO.MediaScanner.ScanFile(string remotePath, MediaScanMode mode);
_adb.Device.IO.MediaScanner.ScanFile(IEnumerable<string> remotePaths, MediaScanMode mode);
_adb.Device.IO.MediaScanner.ScanFile(IEnumerable<string> remotePaths, MediaScanMode mode, CancellationToken cancellationToken);
_adb.Device.IO.MediaScanner.ScanObject(ITransferMessage message, MediaScanMode mode);
EDIT 3:
Found the bad boy, seems "Task.Delay()" only works when used in "async" context. Now the "freeze" is gone, you can move the window until it vanishes. So, in the next update those "freezes" are gone.

---

Little Update:
@squabbi
The next update includes a little editor as suggested by @reynard1. A screenshot can be found as attachment.

Regards,
Sebastian
Attached Thumbnails
Click image for larger version

Name:	editor.png
Views:	18
Size:	55.0 KB
ID:	4949879  
The Following 2 Users Say Thank You to k1ll3r8e For This Useful Post: [ View ] Gift k1ll3r8e Ad-Free
14th February 2020, 01:38 AM |#17  
squabbi's Avatar
Senior Member
Flag Sydney
Thanks Meter: 1,604
 
Donate to Me
More
Quote:
Originally Posted by k1ll3r8e

**snip**

EDIT 3:
Found the bad boy, seems "Task.Delay()" only works when used in "async" context. Now the "freeze" is gone, you can move the window until it vanishes. So, in the next update those "freezes" are gone.

---

Little Update:
@squabbi
The next update includes a little editor as suggested by @reynard1. A screenshot can be found as attachment.

Regards,
Sebastian

Woohoo! Thanks for digging into it! I'm glad you found the culprit haha.

Looking forward to that editor feature! Do you know if it's possible to integrate with Windows Explorer? It's probably a little harder, but something like how WinSCP can drag files from the remote to a local Explorer window.

If that's a little difficult, would this be possible to implement? If I drag and drop a folder/file onto the local pane, it will change directory (and highlight the file if a file was dragged onto the local pane). Just to make it easier to work with folders I have already open.

Still using Windows 7?
The Following User Says Thank You to squabbi For This Useful Post: [ View ] Gift squabbi Ad-Free
14th February 2020, 10:31 AM |#18  
k1ll3r8e's Avatar
OP Senior Member
Flag Delmenhorst
Thanks Meter: 669
 
Donate to Me
More
Quote:
Originally Posted by squabbi

Woohoo! Thanks for digging into it! I'm glad you found the culprit haha.

Looking forward to that editor feature! Do you know if it's possible to integrate with Windows Explorer? It's probably a little harder, but something like how WinSCP can drag files from the remote to a local Explorer window.

If that's a little difficult, would this be possible to implement? If I drag and drop a folder/file onto the local pane, it will change directory (and highlight the file if a file was dragged onto the local pane). Just to make it easier to work with folders I have already open.

Still using Windows 7?

Heyho

Yes its definatly possible, we need to use the so called "VirtualFileDataObject". I tried to implement it some time ago, but doesn't worked as expected. But i'll give it another try.
-
Yup, i still use Win 7 as my main system, but i have some VMs running, like my Debian server which serves the documentation and some test sites for my php projects, a few Android x86 Versions and of course a legit Win10 installation to test If everything works.

*Never change a running system! - Currently my main hdd has a uptime of 2689 days with 216 restarts, i think that tells a lot *

---

Little Update:
As the screenshot shows, the editor is working fine so far. (Added also a few more controls.) The only problem i still have is, that if ADB doesn't run as root you can't edit for example the "build.prop". Because ADB can't write the file directly back, so i need to build a kind of "tmp" file. The next problem will be, that you (the user) have to remount the mount point, before you start editing those "root" and "write protected" files. There is also a problem with "binary" files, because the "RichTextBox" screws up when it gets "binary to ASCII/UTF8" transformed data, which ends in an app freeze... So i have to check on some way if the stream contains really text-data, the file extension doesn't help here much because, you know, linux uses also a lot of txt files without any extension.

Update 2:
Just got the "binary" files running... With a super BIG drawback, 248 kb (4069 rows) text results in 269,364 mb memory utilization by the "RichTextBox". - The prob here is, that the ADB filestream doesn't support seek, so i need to load the whole doc in one, which is quiet to much for the "RichTextBox"...

Update 3:
Soo, i wrote a kind of paging for the "RichTextBox" which works so far really nice, at least when it comes to the memory (even during scrolling, with free running wheel, only 15 - 27 mb increase), there is now a fixed amount of rows (40) which will be shown, all other rows are hidden from the "RichTextBox". This opens a few new issues^^ - Currently there is no selection possible which goes beyond the 40 rows. Also insert and delete are a little challenge because, i need to "overwrite" the default behavior with the some "PreviewKeyDown/etc." events... - Looks all a bit hacky, but lets see if it works in the end.

Update 4:
Well, the "RichTextBox" is out of the race^^ - It triggers events more or less random instead when you expect it, it removes blank lines (for example you have 3x "\n", this gets wrapped to 1x "\n") and the best, you remove "\r" during stream reading and the RTB adds it again, also random (some paragraphs get an "\r" some not). Editing text in a solid way without losing something isn't possible with this thing... So i'll try the "TextBox".

Last Update before release:
So, after testing the "RichTextBox" and "TextBox" i ask myself why those controls are there^^. Both can't edit text with more than a few hundert lines, the "TextBox" is at least much better as the "RichTextBox" but it also screws up when i comes to more as 250 kb text. So the "Editor" will be one of your installed applications. I add a little tool window, which offers the temporary download to the windows temp folder, after that i launch the windows "OpenAs" dialog that you can choose the editor you want. Also, i'll attach a "FileSystemWatcher" to the file that the upload automatically happens when you press save in your editor.

Little Info:
It take a few more days for the "edit" update, because i started also to implement the TAP pattern, this increased for example the download speed by factor 6. But to implement it the right way, i have to rewrite a lot of code in the AndroidCtrl.dll and AndroidCtrlUI.dll.

Regards,
Sebastian
Attached Thumbnails
Click image for larger version

Name:	editor1.png
Views:	8
Size:	68.2 KB
ID:	4950589  
The Following 3 Users Say Thank You to k1ll3r8e For This Useful Post: [ View ] Gift k1ll3r8e Ad-Free
Today, 01:17 AM |#19  
squabbi's Avatar
Senior Member
Flag Sydney
Thanks Meter: 1,604
 
Donate to Me
More
Quote:
Originally Posted by k1ll3r8e

Heyho

Yes its definatly possible, we need to use the so called "VirtualFileDataObject". I tried to implement it some time ago, but doesn't worked as expected. But i'll give it another try.
-
Yup, i still use Win 7 as my main system, but i have some VMs running, like my Debian server which serves the documentation and some test sites for my php projects, a few Android x86 Versions and of course a legit Win10 installation to test If everything works.

*Never change a running system! - Currently my main hdd has a uptime of 2689 days with 216 restarts, i think that tells a lot *

** snip **

That'll be great to see haha, and that does make sense keeping your system as is, if it works, it works.

Thanks for your hard work in checking out what's best for the new edit function. Maybe there's a library out there that can implement a text viewer better? I'm not familiar with WPF libraries, or how many there are.
Post Reply Subscribe to Thread

Tags
adb, explorer, remotedesktop, shell

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes