[APP][ROOT][5.0+][1st Nov 2020] Migrate - custom ROM migration tool

Is Migrate backup and restore working without any errors?


  • Total voters
    901
Search This thread

73sydney

Senior Member
Recently i have had a few migrate backups fail

Like tonight, with errors like:

Zip Read Errors

Errors

<angry red text>
PARSE_APP_MTD: [/sdcard/Migrate/Backup_<date>/Part-x_of_x.zip]
<packagename>.json - end of input at character of
<end of angry red text>

I highly recommend checking your backups after creation by launching a terminal app, changing to the Migrate backup folder (/sdcard/Migrate/Backup<date>) and pasting and running the following command:

for file in *.zip; do echo $file && unzip -l "$file" | awk '{print $1 "\t" $4}' | grep 'json' | grep '^0' | grep -v '/' ; done

It will show you any 0 byte .json files and the zip in which theyre in

If you have access to a PC and a GUI zip client, you can then remove the 0 byte files from the zips

OR

you can manually extract the zips to the usual manual restore folder (/data/local/migrate_cache), by opening a terminal, navigating to /sdcard/Migrate/Backup<date>, and then:

for file in *.zip; do unzip "$file" -d "/data/local/tmp/migrate_cache"; done

then to remove any 0 byte .json files:

find . -name '*.json' -maxdepth 1 -size 0c -print0 | xargs -0 rm

And then Restore via Migrate Helper

Dont ask me why this randomly occurs.....im just suggesting ways to check and work around it. I ended up having to only reinstall 4 apps from scratch in this latest round of weirdness....thankfully for years ive only had a handful of issues with Migrate
 
Last edited:

spinoxda

New member
Oct 2, 2016
4
0
I'd like to perform scheduled backups.
Is there any way to do a manual execution from shell?
Or a way to set up default selections for the the screens, so that just one click on "backup" button would start and unattended run?
thanks
 

73sydney

Senior Member
I'd like to perform scheduled backups.
Is there any way to do a manual execution from shell?
Or a way to set up default selections for the the screens, so that just one click on "backup" button would start and unattended run?
thanks

Automated way to run backups....not that im aware of, and not really in the scope of the app.

You might a way to find the intent/action that fires the backup process with App Manager, on F-Droid or Github here:


It will list all the possible actions, services etc per app

But you would be on your own, and no support :)

The selections are remembered across backups - so you only have to select new apps installed after last backup...
 

73sydney

Senior Member
Will Migrate 4.0 work with Android 13 Roms

If you have Android 13, why not just test it? You cant always rely on someone having done it before you, part of being on XDA is being an adventurer....and at some point every backup app is going to fail

I have used Migrate since inception across at least 3 Android versions, and apart from one small period where it had an issue, its always worked

I would try creating a backup and see if works, you can also check that theres no obvious known issues that occur even when the backup appears to complete successfully, like the one i recently had on Android 12, where some files were 0 byte (for no good reason) in post 1.922 above. It will give you and easy way to test command to check

 

simdwich

New member
Aug 2, 2015
2
1
My experience restoring data on a fresh Lineage 19.1 (Android 12) install, migrating from Android 9, where I backed up using the flasher only method (without backing up permissions):

Flasher complained it could not access any directories ("root node null"). It worked after I manually created the /Migrate folder on internal storage and moved the backup zip there.

Restoring then resulted in a number of errors (something about filename mismatches) for a few apps. I proceeded to open the helper app, which could not find any apps to restore.

I then used the emergency restore method which went through without errors (this step might not be necessary). After that, I re-opened the helper app, which still could not find anything.

I rebooted the device, and finally the helper app was able to find stuff to restore. It finished with errors "[app name].tar.gz was not found" for the same apps that had already produced errors during the regular flasher method I encountered previously.

After restoring, there were some issues with apps crashing, which were resolved after reboot.

Element and Signal did not restore correctly (not totally unexpected) and crashed on opening. Wiping the app data resolved the crashes. No other issues so far.
 

rodken

Senior Member
Jan 11, 2010
1,133
463
My experience restoring data on a fresh Lineage 19.1 (Android 12) install, migrating from Android 9, where I backed up using the flasher only method (without backing up permissions):

Flasher complained it could not access any directories ("root node null"). It worked after I manually created the /Migrate folder on internal storage and moved the backup zip there.

Restoring then resulted in a number of errors (something about filename mismatches) for a few apps. I proceeded to open the helper app, which could not find any apps to restore.

I then used the emergency restore method which went through without errors (this step might not be necessary). After that, I re-opened the helper app, which still could not find anything.

I rebooted the device, and finally the helper app was able to find stuff to restore. It finished with errors "[app name].tar.gz was not found" for the same apps that had already produced errors during the regular flasher method I encountered previously.

After restoring, there were some issues with apps crashing, which were resolved after reboot.

Element and Signal did not restore correctly (not totally unexpected) and crashed on opening. Wiping the app data resolved the crashes. No other issues so far.
In the future -
-- Open Migrate > preferences and scroll down to 'specify path' to ensure that the correct path reads as /data/local/tmp*
-- After creating the migrate-cache folder and copying all contents, reboot before using the Migrate helper.
-- Delete contents of migrate-cache if restore is successful.
 

73sydney

Senior Member
I really dont get whats going on with Migrate lately.....

If it wasnt enough to get 0 byte .json files, tonight weeks later i got some new angry text on restore

"filename not matched"

Turns out the data files (.gz) were all missing on restore

This is becoming a total ballache, and no response from dev on the previous issue.....

I fear it might be time for me to stop championing Migrate and move on.....

Anyways off to setup 139 apps manually ....from scratch.....

(resolved, see later in thread, user error)
 
Last edited:

rodken

Senior Member
Jan 11, 2010
1,133
463
I really dont get whats going on with Migrate lately.....

If it wasnt enough to get 0 byte .json files, tonight weeks later i got some new angry text on restore

"filename not matched"

Turns out the data files (.gz) were all missing on restore

This is becoming a total ballache, and no response from dev on the previous issue.....

I fear it might be time for me to stop championing Migrate and move on.....

Anyways off to setup 139 apps manually ....from scratch.....
No response from the developer?
-- I hope that this project won't fall into the 'abandonware' category.
 

73sydney

Senior Member
No response from the developer?
-- I hope that this project won't fall into the 'abandonware' category.

Its been a while since ive seen any action, i ran into him on Telegram a few months ago as i made a joke in a channel and he commented and i mentioned my long term use of Migrate. He's had "build your own" code for v5v on his github for months, and ive tried a couple of compiled test builds, none of which managed to complete a backup cycle. His channel is active on Telegram, but im not sure what his involvement is there (i mostly skim stuff in 10 minutes at the end of my tech day), and i respect that people have busy lives

So i dont know what that means, but its odd ive just started having the issues ive had in the last few months, im not running custom ROM's, im using bone stock Samsung A12, and it while i can understand the first issue i had (0 byte .json files not triggering an error when backing up), not registering that the .gz of userdata failed is a whole nother thing

Except for a small period where there was a serious bug about 2 years back, ive always recommended Migrate at every chance i got, and at one point it was central in my signature....

At this point im going to use Alpha Backup or even try my old MyBackup Pro (that i left for Migrate years ago)
 
Last edited:
  • Like
Reactions: rodken

Kippy51

Senior Member
Mar 11, 2015
1,391
752
Haren (Ems)
@73sydney
"Turns out the data files (.gz) were all missing on restore"
Where they really missing or just not restored? I had the same problem restoring my 165 apps from A11 to A13. It turned out they were in the backup, just not put in the cache-folder from Migrate. With TotalCommander I extracted the backup-zip to the cache-folder and using the Helper app I installed the apps, without the permissions. To be on the safe side I restored 10 at a time, but I had no further problems.
 
  • Like
Reactions: rodken

73sydney

Senior Member
Swift Backup is worth a look too

Cheers, every time i see your name for some reason in my head i think brackenhill mob sounds like it should be gang from south Boston, but then i realised after googling bracken hill is a place in Yorkshire, which then made me think of Yorkshire Gold (my fave tea, or "Parky Tea" as we call it in my house after Mr Parkinson), and then i got depressed because i had to give up caffeine a decade or more ago

I still miss my Yoskshire Gold....
 

73sydney

Senior Member
@73sydney
"Turns out the data files (.gz) were all missing on restore"
Where they really missing or just not restored? I had the same problem restoring my 165 apps from A11 to A13. It turned out they were in the backup, just not put in the cache-folder from Migrate. With TotalCommander I extracted the backup-zip to the cache-folder and using the Helper app I installed the apps, without the permissions. To be on the safe side I restored 10 at a time, but I had no further problems.

Fair question to ask, but no, mine were totes missing

(resolved, see later in thread, user error)
 
Last edited:

geos_

Member
Dec 28, 2018
42
24
i have older unrooted Zenfone 3 Zoom. will this app be able to backup and restore Telegram's secret chats?
 

Top Liked Posts

  • There are no posts matching your filters.
  • 4
    My experience restoring data on a fresh Lineage 19.1 (Android 12) install, migrating from Android 9, where I backed up using the flasher only method (without backing up permissions):

    Flasher complained it could not access any directories ("root node null"). It worked after I manually created the /Migrate folder on internal storage and moved the backup zip there.

    Restoring then resulted in a number of errors (something about filename mismatches) for a few apps. I proceeded to open the helper app, which could not find any apps to restore.

    I then used the emergency restore method which went through without errors (this step might not be necessary). After that, I re-opened the helper app, which still could not find anything.

    I rebooted the device, and finally the helper app was able to find stuff to restore. It finished with errors "[app name].tar.gz was not found" for the same apps that had already produced errors during the regular flasher method I encountered previously.

    After restoring, there were some issues with apps crashing, which were resolved after reboot.

    Element and Signal did not restore correctly (not totally unexpected) and crashed on opening. Wiping the app data resolved the crashes. No other issues so far.
    In the future -
    -- Open Migrate > preferences and scroll down to 'specify path' to ensure that the correct path reads as /data/local/tmp*
    -- After creating the migrate-cache folder and copying all contents, reboot before using the Migrate helper.
    -- Delete contents of migrate-cache if restore is successful.
    3
    Will Migrate 4.0 work with Android 13 Roms
    Works for me on Android 13
    2
    Will Migrate 4.0 work with Android 13 Roms

    If you have Android 13, why not just test it? You cant always rely on someone having done it before you, part of being on XDA is being an adventurer....and at some point every backup app is going to fail

    I have used Migrate since inception across at least 3 Android versions, and apart from one small period where it had an issue, its always worked

    I would try creating a backup and see if works, you can also check that theres no obvious known issues that occur even when the backup appears to complete successfully, like the one i recently had on Android 12, where some files were 0 byte (for no good reason) in post 1.922 above. It will give you and easy way to test command to check

    2
    MiGREAT works on the kitchen sink also. It's a superb tool(y)(y)(y)
    2
    At this point im going to use Alpha Backup or even try my old MyBackup Pro (that i left for Migrate years ago)
    Swift Backup is worth a look too
  • 603
    Jumping from one custom ROM to another can be a challenging task. So I decided to make an app to simplify the process.

    Introducing

    Migrate!

    Migrate-GPE (v4.0) is on Google Play
    It was temporarily suspended. Read more here: https://forum.xda-developers.com/showpost.php?p=82230451&postcount=1273

    Download from Google Play Store
    Download from AndroidFileHost
    Download from XDA labs (removed as XDA Labs has shutdown)

    CODEBASE ON GITHUB


    migrate_1.png


    With Migrate, you can back up:
    1. App apk
    2. App data
    3. App runtime permissions
    4. Contacts (as vcf file)
    5. SMS messages
    6. Call logs
    7. Current ROM screen DPI
    8. Current default keyboard option
    9. App installer (Migrate 3.0)
    10. Adb state (Migrate 3.0)
    11. Font scale (Migrate 3.0)

    ... everything in one app.

    migrate_3.png

    migrate_4.png


    Ok. So its just another of those backup and restore apps, right?

    Sort of... but with a twist.
    Restoring everything requires flashing the backup via TWRP.

    ALTERNATELY
    You can use the Migrate Flasher app (if your phone does not have a compatible recovery, or your recovery is broken).

    If you are flashing via TWRP, see the below screenshot.

    migrate_recovery.png


    All your apps and data are restored to a directory under /data/balti.migrate, /data/local/tmp/migrate_cache (from v2.0+) and a helper app is injected into the system.. Once you boot up your new ROM, the app will spring up, ask for root permission and do all the job while you can relax.

    migrate_helper.png


    IF YOU ARE USING MIGRATE FLASHER
    The process is similar. Only difference is Migrate Helper is not injected into system, but installed as a user app. There is no use of recovery environment.
    Once Migrate Helper is installed, just press RESTORE APPS AND DATA and proceed as usual.

    Please see the HOW TO FLASH section on Migrate Flasher for complete illustration.

    IF you have already flashed using TWRP (with no errors) then the above sections about Migrate Flasher is not applicable.

    Donations:
    Well, the app already has ads, so you need not explicitly donate anything :p
    But if you are feeling reaalllyyy grateful for the app, you can donate here: paypal.me/sayantanrc
    Thanks.

    Migrate Gemini (v4.0) release

    Hello everyone.
    Migrate 4.0 is now released.

    There are 2 editions of Migrate.

    Migrate NG: This is the Non-Google and Beta edition. It is not published on Google Play store. It will continue having in-app updates. Moreover new features will be tested on these builds first. People who did not like firebase analytics, this edition is for you as it does not have firebase SDK anymore. However, Google admob ads are still present.

    Download link (NG): https://www.androidfilehost.com/?fid=10763459528675575696

    Migrate GPE: This version is Google Play Edition. It is the stable version. It does not have in-app updates anymore as it violated Google Play policies.
    Download on Google Play: https://play.google.com/store/apps/details?id=balti.migrate

    Coming from alpha 3.1, if you update via the in-app updater, you will be switched to the Non-Google Edition.

    Thank you.

    Want to translate the app?
    The strings.xml file is available here: https://github.com/SayantanRC/translations

    Telegram group!
    We now have a Telegram group for migrate:
    http://t.me/migrateApp
    This is not to rival XDA but only another form of communication between users and me :)
    Thanks.
    144
    FAQ:

    Click here for Troubleshooting options


    If you are facing issues while flashing, refer to this video: https://youtu.be/0Cp6TNayXnM
    Full manual extract guide: https://forum.xda-developers.com/showpost.php?p=82198185&postcount=1269

    How is this app different from Titanium backup?
    This app is a ROM migration tool. It not only backs up app and data, but also contacts, SMS, call logs etc. Moreover, the restoration process also involves flashing via TWRP. Titanium backup is an excellent app. Migrate just views the restoration process differently.

    In recent times however, Migrate Flasher app can be used to flash a zip even if you dont have a compatible TWRP recovery. Everything is almost the same other than just the flashing part, which is done from the Flasher app.

    Do I need to download the Migrate app to restore the apps?
    No. Only your ROM needs to be rooted. The restoration process is handled by a helper app, which deletes itself after restoring everything.

    Is it necessary to use Magisk?
    The app works best with Magisk. Other options like Lineage OS su addon causes weird errors like incomplete file copy, etc.

    All apps are restored from recovery?
    No. That is not possible. Because Recovery is a completely different environment from Android. Think of it as installing a Windows game from a Linux distro. Instead from recovery, an app is injected into system which takes up the restore process when android boots up. Most things are automated, so you need not worry much.

    A quick word on SD card support...
    SD card support in v2.0 requires a Magisk module "ExSDCard Access Enabler", made by developer Romain. According to him, this module requires FUSE support on ROMs. FUSE is supported on some ROMs as a legacy feature as Android has moved away from FUSE to SDCARDFS. As such, the current implementation may or may not work based on your device and ROM.
    If it does not work for you, please wait until either the module developer rewrites the module :p or I find a different implementation :) Thanks for your patience.


    From version 5.0+ (Not yet released, available on GitHub)
    Users will be able to natively store backups on SD Cards, especially on Android 11+.
    For lower Android versions, Storage Access Framework is used (along with root access) to store backups on SD Cards as well as on USB-OTG.
    57
    Just posting this for everyone to know.

    An alpha build of Migrate 3.0 should be ready by the end of this month. The version should support Android 10, and will come with new features.

    In the main app, backup and all new features are fully operational. But the helper app is not ready yet. I am working on it. That should be ready by this month.

    Please hold on a little. Thank you for your support. ?
    54
    Changelog

    Version 4.0 [1st November 2020]

    # Compatibility fixes for Android 11
    # Add Message center to display important announcements.
    # Migrate helper now allows to retry failed apk installs
    # Add a "Migrate flasher only" backup option
    # Save app icons as png files
    # Show number of apps selected for backup in Extra backups screen
    # Keep screen on when calculating backup size
    # Save last state of selection in extras
    # Don't remount /data partition in system test
    # Many under the hood changes

    Version 3.1.1 [25thApril 2020]

    # Fix issue in v3.1 where restore would not be possible for apps with unsupported characters in the name.

    Version 3.1 [24thApril 2020]

    # Probable fix to data restore problems
    # Probable fix to helper crashing during loading apps
    # Record raw list of files in a backup
    # Many other minor improvements

    Use v3.1.1 above

    Version 3.0.3 [18thApril 2020]

    #. Slightly tweaked logo.
    #. Probable fix to No metadata found after flashing.

    Version 3.0.1 [9thApril 2020]

    #. Bug fixes
    #. Added translations

    Version 3.0 [20th March 2020]

    #. The app is rewritten in Kotlin
    #. Many new preferences have been added.
    #. Helper app is now available on Google Play.
    #. Many fixes in recovery. Support for SAR and Android 10.
    #. Added options for backup of app installer, font scale, adb state
    #. Set max zip size in preferences.
    #. Verification of zips after backup is complete.
    #. Many more changes and improvements…

    Version 2.1 [15th Jan]

    # Minor errors are now suppressed, but recorded in log.
    # Fix icon retention form previous backup
    # Fix wrong backup duration display.
    # Fixed a bug in error correction mechanism.
    # Probable fix for some apps not being backed up.
    # Fix for error 6 in TWRP for some locales.
    # Fixed a long standing wrong /data space calculation in TWRP.
    # Migrate helper no longer uninstalls app before installing.
    # Added Telegram group link!

    Version 2.0.1 [30th Dec] (christmas_beta)

    # Fixed a probable crash in Migrate helper (needs a magisk module)

    Version 2.0 [26th Dec] (christmas_beta)

    # Experimental SD card support (needs a magisk module)
    (please read the previous post)
    # Support for split apks
    # "Abort" option in Migrate helper
    # "Temporary disable" in Migrate helper
    # Apks are now unpacked under /data/local/tmp/migrate_cache
    (probable fix to some devices unable to install apks)
    # Each backup zip now has a busybox file as fallback

    Version 1.2 [12th Nov] (beta)

    # Completely avoid "dumpsys" to backup app permissions
    # Now app data in tar.gz is unpacked in /data/data
    (probable fix to "Operation not permitted" issues in Google Pixel and some other devices)

    Version 1.1.1 [10th Nov] (beta)

    v1.1 (skipping upload)

    # [New] Backup verification before zipping
    # Major code refactoring and under-the-hood changes
    # CANCEL button now kills the backup process
    # Fix to "set_progress()" in recovery
    # Probable fix to "Failed to change locale for db"
    # Probable fix to missing updater-script
    # Probable fix to some "Operation not permitted" issues
    # Remove illegal characters from backup name
    # "Share scripts" option should now work properly.

    v1.1.1

    # Fix to a no-closing-quote issue in helper

    Version 1.0.5 [7th Nov] (beta)
    # Probable fix to "android.os.TransactionTooLargeException"
    # Better in app protection to prevent zips from not being made
    # App version is now written in logs
    # App now targets Oreo instead of Pie to potentially avoid some crashes
    # Fixed unexpected ( in restore process

    Version 1.0.4 [5th Nov] (beta)
    # Improvements to logging
    # Ignore tar socket warnings
    # New update-binary to prevent TWRP error 3
    # Should fix a null String issue while reading backup data

    Version 1.0.3 (beta)
    # Fixed log sharing
    # Share script file in log
    # Blacklist Android system from system app backup list (prevents tar socket error)

    Version 1.0 (beta)
    # Initial release
    32
    Hi @SayantanRC ,
    actually I'm a TWRP developer and i'm working on a lot of devices and recently a bit of users starting to tell me that they can't flash their migrate backups, so I found myself investigating a little and at the end i found the problem: in the migrate zips that migrate app creates, there are some commands that use busybox that actually is deprecated and not more used in TWRP. Can you please use generic commands or make a choice to use toybox instead of busybox?
    For example this command:

    run_program("/sbin/busybox", "umount", "/data");

    Can be replaced from this one:

    unmount("/data");

    Or another option is to include a fully busybox set of binaries in the migrate zips so they will be independent. If you need some help or explanations contact me.

    P.s. i already sent also a mail to your play store supprot email.