• Introducing XDA Computing: Discussion zones for Hardware, Software, and more!    Check it out!

[APP][ROOT][5.0+][1st Nov 2020] Migrate - custom ROM migration tool

Is Migrate backup and restore working without any errors?


  • Total voters
    825
Search This thread

SayantanRC

Senior Member
Jan 26, 2016
389
1,593
Kharagpur
Jumping from one custom ROM to another can be a challenging task. So I decided to make an app to simplify the process.

Introducing
Migrate!

Migrate-GPE (v4.0) is on Google Play
It was temporarily suspended. Read more here: https://forum.xda-developers.com/showpost.php?p=82230451&postcount=1273

Download from Google Play Store
Download from AndroidFileHost
Download from XDA labs

migrate_1.png


With Migrate, you can back up:
  1. App apk
  2. App data
  3. App runtime permissions
  4. Contacts (as vcf file)
  5. SMS messages
  6. Call logs
  7. Current ROM screen DPI
  8. Current default keyboard option
  9. App installer (Migrate 3.0)
  10. Adb state (Migrate 3.0)
  11. Font scale (Migrate 3.0)

... everything in one app.

migrate_3.png

migrate_4.png


Ok. So its just another of those backup and restore apps, right?

Sort of... but with a twist.
Restoring everything requires flashing the backup via TWRP.

migrate_recovery.png


All your apps and data are restored to a directory under /data/balti.migrate, /data/local/tmp/migrate_cache (from v2.0+) and a helper app is injected into the system.. Once you boot up your new ROM, the app will spring up, ask for root permission and do all the job while you can relax.

migrate_helper.png


Donations:
Well, the app already has ads, so you need not explicitly donate anything :p
But if you are feeling reaalllyyy grateful for the app, you can donate here: paypal.me/sayantanrc
Thanks.

Migrate Gemini (v4.0) release

Hello everyone.
Migrate 4.0 is now released.

There are 2 editions of Migrate.

Migrate NG: This is the Non-Google and Beta edition. It is not published on Google Play store. It will continue having in-app updates. Moreover new features will be tested on these builds first. People who did not like firebase analytics, this edition is for you as it does not have firebase SDK anymore. However, Google admob ads are still present.

Download link (NG): https://www.androidfilehost.com/?fid=10763459528675575696

Migrate GPE: This version is Google Play Edition. It is the stable version. It does not have in-app updates anymore as it violated Google Play policies.
Download on Google Play: https://play.google.com/store/apps/details?id=balti.migrate

Coming from alpha 3.1, if you update via the in-app updater, you will be switched to the Non-Google Edition.

Thank you.

Want to translate the app?
The strings.xml file is available here: https://github.com/SayantanRC/translations

Telegram group!
We now have a Telegram group for migrate:
http://t.me/migrateApp
This is not to rival XDA but only another form of communication between users and me :)
Thanks.
 

Attachments

  • migrate_4.0_NG.apk
    11.2 MB · Views: 2,502
  • migrate_4.0_GPE.apk
    11.4 MB · Views: 964
  • MigrateHelper.apk
    5.1 MB · Views: 2,102
Last edited:

SayantanRC

Senior Member
Jan 26, 2016
389
1,593
Kharagpur
FAQ:

Click here for Troubleshooting options

If you are facing issues while flashing, refer to this video: https://youtu.be/0Cp6TNayXnM
Full manual extract guide: https://forum.xda-developers.com/showpost.php?p=82198185&postcount=1269

How is this app different from Titanium backup?
This app is a ROM migration tool. It not only backs up app and data, but also contacts, SMS, call logs etc. Moreover, the restoration process also involves flashing via TWRP. Titanium backup is an excellent app. Migrate just views the restoration process differently.

Do I need to download the Migrate app to restore the apps?
No. Only your ROM needs to be rooted. The restoration process is handled by a helper app, which deletes itself after restoring everything.

Is it necessary to use Magisk?
The app works best with Magisk. Other options like Lineage OS su addon causes weird errors like incomplete file copy, etc.

All apps are restored from recovery?
No. That is not possible. Because Recovery is a completely different environment from Android. Think of it as installing a Windows game from a Linux distro. Instead from recovery, an app is injected into system which takes up the restore process when android boots up. Most things are automated, so you need not worry much.

A quick word on SD card support...
SD card support in v2.0 requires a Magisk module "ExSDCard Access Enabler", made by developer Romain. According to him, this module requires FUSE support on ROMs. FUSE is supported on some ROMs as a legacy feature as Android has moved away from FUSE to SDCARDFS. As such, the current implementation may or may not work based on your device and ROM.
If it does not work for you, please wait until either the module developer rewrites the module :p or I find a different implementation :) Thanks for your patience.
 
Last edited:

SayantanRC

Senior Member
Jan 26, 2016
389
1,593
Kharagpur
Changelog

Version 4.0 [1st November 2020]

# Compatibility fixes for Android 11
# Add Message center to display important announcements.
# Migrate helper now allows to retry failed apk installs
# Add a "Migrate flasher only" backup option
# Save app icons as png files
# Show number of apps selected for backup in Extra backups screen
# Keep screen on when calculating backup size
# Save last state of selection in extras
# Don't remount /data partition in system test
# Many under the hood changes

Version 3.1 [25thApril 2020]

# Fix issue in v3.1 where restore would not be possible for apps with unsupported characters in the name.

Version 3.1 [24thApril 2020]

# Probable fix to data restore problems
# Probable fix to helper crashing during loading apps
# Record raw list of files in a backup
# Many other minor improvements

Use v3.1.1 above

Version 3.0.3 [18thApril 2020]

#. Slightly tweaked logo.
#. Probable fix to No metadata found after flashing.

Version 3.0.1 [9thApril 2020]

#. Bug fixes
#. Added translations

Version 3.0 [20th March 2020]

#. The app is rewritten in Kotlin
#. Many new preferences have been added.
#. Helper app is now available on Google Play.
#. Many fixes in recovery. Support for SAR and Android 10.
#. Added options for backup of app installer, font scale, adb state
#. Set max zip size in preferences.
#. Verification of zips after backup is complete.
#. Many more changes and improvements…

Version 2.1 [15th Jan]

# Minor errors are now suppressed, but recorded in log.
# Fix icon retention form previous backup
# Fix wrong backup duration display.
# Fixed a bug in error correction mechanism.
# Probable fix for some apps not being backed up.
# Fix for error 6 in TWRP for some locales.
# Fixed a long standing wrong /data space calculation in TWRP.
# Migrate helper no longer uninstalls app before installing.
# Added Telegram group link!

Version 2.0.1 [30th Dec] (christmas_beta)

# Fixed a probable crash in Migrate helper (needs a magisk module)

Version 2.0 [26th Dec] (christmas_beta)

# Experimental SD card support (needs a magisk module)
(please read the previous post)
# Support for split apks
# "Abort" option in Migrate helper
# "Temporary disable" in Migrate helper
# Apks are now unpacked under /data/local/tmp/migrate_cache
(probable fix to some devices unable to install apks)
# Each backup zip now has a busybox file as fallback

Version 1.2 [12th Nov] (beta)

# Completely avoid "dumpsys" to backup app permissions
# Now app data in tar.gz is unpacked in /data/data
(probable fix to "Operation not permitted" issues in Google Pixel and some other devices)

Version 1.1.1 [10th Nov] (beta)

v1.1 (skipping upload)

# [New] Backup verification before zipping
# Major code refactoring and under-the-hood changes
# CANCEL button now kills the backup process
# Fix to "set_progress()" in recovery
# Probable fix to "Failed to change locale for db"
# Probable fix to missing updater-script
# Probable fix to some "Operation not permitted" issues
# Remove illegal characters from backup name
# "Share scripts" option should now work properly.

v1.1.1

# Fix to a no-closing-quote issue in helper

Version 1.0.5 [7th Nov] (beta)
# Probable fix to "android.os.TransactionTooLargeException"
# Better in app protection to prevent zips from not being made
# App version is now written in logs
# App now targets Oreo instead of Pie to potentially avoid some crashes
# Fixed unexpected ( in restore process

Version 1.0.4 [5th Nov] (beta)
# Improvements to logging
# Ignore tar socket warnings
# New update-binary to prevent TWRP error 3
# Should fix a null String issue while reading backup data

Version 1.0.3 (beta)
# Fixed log sharing
# Share script file in log
# Blacklist Android system from system app backup list (prevents tar socket error)

Version 1.0 (beta)
# Initial release
 
Last edited:

SayantanRC

Senior Member
Jan 26, 2016
389
1,593
Kharagpur
Some plaguing bugs on some devices :(


"dumpsys not found"
(Severity: mild. Only permissions are not backed up)
Mainly in ASUS devices. Probable solution is to not use dumpsys at all.

Probably fixed in v1.2

"tar: can't execute 'gzip': Permission denied"
(Severity: severe. Do not restore this backup)
I still have no idea why this is happening. Probable solution is use something other than tar.gz


Data files cannot be moved during restore. "Operation not permitted"
(Severity: severe)
No idea yet. Prominent in Google Pixel devices and some Xiaomi devices

Probably partially or completely fixed in v1.2

Error 7 (or 6) in TWRP
(Severity: severe)

This issue which was supposedly fixed in v1.1.1 seems to occur again on few devices
Since I cannot reproduce the issue and the number of error reports are pretty small, please send the recovery.log from TWRP:
- Goto Advanced -> Terminal in TWRP
OR
open an adb shell inside TWRP
- Type the following and press enter: cp /tmp/recovery.log /sdcard/
- Reboot and upload the recovery.log file (from internal sd card) here.



Stuck at calculating size.
(Severity: severe)

This is being reported multiple times. I am still unable to reproduce the issue on my device.
That being said, I will be adding new controls to skip size checking of its stuck on any device, in the next release. Also, I will add more fine logging to see what is actually happening behind the scenes, which cannot be detected by a logcat.

Probably fixed in v2.0

Pocophone TWRP recovery has issues
(Severity: severe)

I need someone to do the following and then upload the file:/sdcard/test_log.log
- TWRP main menu -> Mount -> Data and System should be mounted
- TWRP main menu -> Advanced -> Terminal
OR
open an adb shell inside TWRP
- Type the following commands:

getprop > /sdcard/test_log.log
echo " " >> /sdcard/test_log.log
df /data >> /sdcard/test_log.log
echo " " >> /sdcard/test_log.log
df -k /data >> /sdcard/test_log.log
echo " " >> /sdcard/test_log.log
df -h /data >> /sdcard/test_log.log
echo " " >> /sdcard/test_log.log
df /system >> /sdcard/test_log.log

The following should work as an alternative:
You have to manually do the following:
1. Open the backup zip file
2. Edit the "package-data.txt" file
3. Put the "data_required_size" as "0"
4. Save the file.
5. Repack the zip file.

"Segmentation fault"
(Severity: unknown)
Mainly in Xiaomi devices. I cannot reproduce it. Neither do I know how much this affects the backup. Please restore a backup and tell me if it works.
 
Last edited by a moderator:

SayantanRC

Senior Member
Jan 26, 2016
389
1,593
Kharagpur
Quite promising
So what does the helper app really do? Move the apps and data from /data/balti.migrate to their respective directories? How does that work in a booted up ROM?

The helper app installs the apks using package manager and then unpacks the data of the respective apps in /data/data. Finally it restores all the apps runtime permissions.
 

m0d

Senior Member
Nov 3, 2010
1,052
419
The helper app installs the apks using package manager and then unpacks the data of the respective apps in /data/data. Finally it restores all the apps runtime permissions.

Does it handle folder permissions when there are changes in app uid for data restore
when pm installs app, app uid can be different than before, so there might be folder permission discrepencies?

I.E.
Code:
#!/bin/sh
if [ "$1" != "" ]; 
then 	
#The folders in /data/data each have a different owner. The next line gets the owner and the strips the whitespace 	

appusername=$(stat -c "%U" /data/data/$1 | xargs) 	
#The next line constructs the chown command to repair the permissions for all the files in the target folder 	
fixperm="chown -R $appusername:$appusername /data/data/$1/*" 	
#Display the command to the user 	
echo "$fixperm" 	
#Uncomment the next line if you would like the script to fix the permission 	
eval $fixperm 
else echo "Missing target folder parameter" 
fi
 
Last edited:

SayantanRC

Senior Member
Jan 26, 2016
389
1,593
Kharagpur
Does it handle folder permissions when there are changes in app uid for data restore
when pm installs app, app uid can be different than before, so there might be folder permission discrepencies?

I.E.
Code:
#!/bin/sh
if [ "$1" != "" ]; 
then 	
#The folders in /data/data each have a different owner. The next line gets the owner and the strips the whitespace 	

appusername=$(stat -c "%U" /data/data/$1 | xargs) 	
#The next line constructs the chown command to repair the permissions for all the files in the target folder 	
fixperm="chown -R $appusername:$appusername /data/data/$1/*" 	
#Display the command to the user 	
echo "$fixperm" 	
#Uncomment the next line if you would like the script to fix the permission 	
eval $fixperm 
else echo "Missing target folder parameter" 
fi

It takes care of the new uid for the app. No problem with that.
 

shadowstep

Forum Moderator
Staff member
Jun 6, 2014
5,630
11,695
30
Ambala
OnePlus 5
Does it handle folder permissions when there are changes in app uid for data restore
when pm installs app, app uid can be different than before, so there might be folder permission discrepencies?

Do you have an idea if Titanium Backup and/or Swift Backup take care of app uid changes during data restore to avoid folder permission discrepancies?
 

HuyPui

Member
Sep 3, 2017
16
0
Hanoi
Can i use this to jump between like for example MIUI 10 China Stable(with root) to 9.0 Pie custom roms ? Using OrangeFox TWRP,Redmi Note 4x(mido)
 

Top Liked Posts

  • There are no posts matching your filters.
  • 4
    Some apps don't work after restoration. They just crash when trying to open them. Anyone else with this problem?

    Make a note of which apps dont seamlessly run after restore, and then untick the centre column for each, the best part is then from then on Migrate will remember those preferences on each successive backup

    As a general guide, leave column unticked for:

    Any banking apps
    Samsung Health
    Samsung Health monitor
    Samsung Gear Wearables apps
    1
    I hope you'll accept my apologies at questioning whether you have magisk installed and working correctly, because going on the screens shared youre right, you must have a device where its installed to recovery rather than ramdisk. I just cannot currently explain why it is not working for those two apps, if its working for ACCA and the TWRP app, it should work for all...you would think

    What device do you have

    And

    What happens if you manually try creating the path via terminal as root:

    Code:
    su <enter>
    mdkir -p /data/local/tmp/migrate_tmp <enter>

    Hi,

    I've just tried what you've suggested and it still throws the same error. I have used Termux app to create the path via terminal. Is this the right approach?

    Any other ideas?

    Btw, you suggested creating a migrate_tmp folder however on the error shows migrate_cache. Is this okay?

    My device is a Xiaomi Redmi Note 8 (Ginkgo), running Paranoid ROM (Android 10)

    Thank you

    UPDATE: I tried the code with migrate_cache (instead of migrate_tmp) and it also throws an 'operation not permitted'.. even though it was granted root permissions.

    UPDATE#2: I've tried creating the folder with a root explorer and it also fails with 'operation not permitted'.. I can create any other folder though. It's just not possible to create that specific folder name. I've also tried creating a folder with other name and rename it to 'migrate_cache' and it fails saying unable to rename.. is it possible to use another folder name instead?

    UPDATE#3: I copied the contents from the backup zip into one of the other folders I created, changed the path to migrate_cache on the helper app preferences to that folder I created and everything worked!!

    very strange error/behaviour. But I'm glad it worked.

    THANK YOU!
    1
    Some apps don't work after restoration. They just crash when trying to open them. Anyone else with this problem?
    If you mean bankapps then yes. That's for security I guess. Also Slack wants you to log in again. For me it's not a problem. Safety first!
  • 588
    Jumping from one custom ROM to another can be a challenging task. So I decided to make an app to simplify the process.

    Introducing
    Migrate!

    Migrate-GPE (v4.0) is on Google Play
    It was temporarily suspended. Read more here: https://forum.xda-developers.com/showpost.php?p=82230451&postcount=1273

    Download from Google Play Store
    Download from AndroidFileHost
    Download from XDA labs

    migrate_1.png


    With Migrate, you can back up:
    1. App apk
    2. App data
    3. App runtime permissions
    4. Contacts (as vcf file)
    5. SMS messages
    6. Call logs
    7. Current ROM screen DPI
    8. Current default keyboard option
    9. App installer (Migrate 3.0)
    10. Adb state (Migrate 3.0)
    11. Font scale (Migrate 3.0)

    ... everything in one app.

    migrate_3.png

    migrate_4.png


    Ok. So its just another of those backup and restore apps, right?

    Sort of... but with a twist.
    Restoring everything requires flashing the backup via TWRP.

    migrate_recovery.png


    All your apps and data are restored to a directory under /data/balti.migrate, /data/local/tmp/migrate_cache (from v2.0+) and a helper app is injected into the system.. Once you boot up your new ROM, the app will spring up, ask for root permission and do all the job while you can relax.

    migrate_helper.png


    Donations:
    Well, the app already has ads, so you need not explicitly donate anything :p
    But if you are feeling reaalllyyy grateful for the app, you can donate here: paypal.me/sayantanrc
    Thanks.

    Migrate Gemini (v4.0) release

    Hello everyone.
    Migrate 4.0 is now released.

    There are 2 editions of Migrate.

    Migrate NG: This is the Non-Google and Beta edition. It is not published on Google Play store. It will continue having in-app updates. Moreover new features will be tested on these builds first. People who did not like firebase analytics, this edition is for you as it does not have firebase SDK anymore. However, Google admob ads are still present.

    Download link (NG): https://www.androidfilehost.com/?fid=10763459528675575696

    Migrate GPE: This version is Google Play Edition. It is the stable version. It does not have in-app updates anymore as it violated Google Play policies.
    Download on Google Play: https://play.google.com/store/apps/details?id=balti.migrate

    Coming from alpha 3.1, if you update via the in-app updater, you will be switched to the Non-Google Edition.

    Thank you.

    Want to translate the app?
    The strings.xml file is available here: https://github.com/SayantanRC/translations

    Telegram group!
    We now have a Telegram group for migrate:
    http://t.me/migrateApp
    This is not to rival XDA but only another form of communication between users and me :)
    Thanks.
    140
    FAQ:

    Click here for Troubleshooting options

    If you are facing issues while flashing, refer to this video: https://youtu.be/0Cp6TNayXnM
    Full manual extract guide: https://forum.xda-developers.com/showpost.php?p=82198185&postcount=1269

    How is this app different from Titanium backup?
    This app is a ROM migration tool. It not only backs up app and data, but also contacts, SMS, call logs etc. Moreover, the restoration process also involves flashing via TWRP. Titanium backup is an excellent app. Migrate just views the restoration process differently.

    Do I need to download the Migrate app to restore the apps?
    No. Only your ROM needs to be rooted. The restoration process is handled by a helper app, which deletes itself after restoring everything.

    Is it necessary to use Magisk?
    The app works best with Magisk. Other options like Lineage OS su addon causes weird errors like incomplete file copy, etc.

    All apps are restored from recovery?
    No. That is not possible. Because Recovery is a completely different environment from Android. Think of it as installing a Windows game from a Linux distro. Instead from recovery, an app is injected into system which takes up the restore process when android boots up. Most things are automated, so you need not worry much.

    A quick word on SD card support...
    SD card support in v2.0 requires a Magisk module "ExSDCard Access Enabler", made by developer Romain. According to him, this module requires FUSE support on ROMs. FUSE is supported on some ROMs as a legacy feature as Android has moved away from FUSE to SDCARDFS. As such, the current implementation may or may not work based on your device and ROM.
    If it does not work for you, please wait until either the module developer rewrites the module :p or I find a different implementation :) Thanks for your patience.
    57
    Just posting this for everyone to know.

    An alpha build of Migrate 3.0 should be ready by the end of this month. The version should support Android 10, and will come with new features.

    In the main app, backup and all new features are fully operational. But the helper app is not ready yet. I am working on it. That should be ready by this month.

    Please hold on a little. Thank you for your support. ?
    53
    Changelog

    Version 4.0 [1st November 2020]

    # Compatibility fixes for Android 11
    # Add Message center to display important announcements.
    # Migrate helper now allows to retry failed apk installs
    # Add a "Migrate flasher only" backup option
    # Save app icons as png files
    # Show number of apps selected for backup in Extra backups screen
    # Keep screen on when calculating backup size
    # Save last state of selection in extras
    # Don't remount /data partition in system test
    # Many under the hood changes

    Version 3.1 [25thApril 2020]

    # Fix issue in v3.1 where restore would not be possible for apps with unsupported characters in the name.

    Version 3.1 [24thApril 2020]

    # Probable fix to data restore problems
    # Probable fix to helper crashing during loading apps
    # Record raw list of files in a backup
    # Many other minor improvements

    Use v3.1.1 above

    Version 3.0.3 [18thApril 2020]

    #. Slightly tweaked logo.
    #. Probable fix to No metadata found after flashing.

    Version 3.0.1 [9thApril 2020]

    #. Bug fixes
    #. Added translations

    Version 3.0 [20th March 2020]

    #. The app is rewritten in Kotlin
    #. Many new preferences have been added.
    #. Helper app is now available on Google Play.
    #. Many fixes in recovery. Support for SAR and Android 10.
    #. Added options for backup of app installer, font scale, adb state
    #. Set max zip size in preferences.
    #. Verification of zips after backup is complete.
    #. Many more changes and improvements…

    Version 2.1 [15th Jan]

    # Minor errors are now suppressed, but recorded in log.
    # Fix icon retention form previous backup
    # Fix wrong backup duration display.
    # Fixed a bug in error correction mechanism.
    # Probable fix for some apps not being backed up.
    # Fix for error 6 in TWRP for some locales.
    # Fixed a long standing wrong /data space calculation in TWRP.
    # Migrate helper no longer uninstalls app before installing.
    # Added Telegram group link!

    Version 2.0.1 [30th Dec] (christmas_beta)

    # Fixed a probable crash in Migrate helper (needs a magisk module)

    Version 2.0 [26th Dec] (christmas_beta)

    # Experimental SD card support (needs a magisk module)
    (please read the previous post)
    # Support for split apks
    # "Abort" option in Migrate helper
    # "Temporary disable" in Migrate helper
    # Apks are now unpacked under /data/local/tmp/migrate_cache
    (probable fix to some devices unable to install apks)
    # Each backup zip now has a busybox file as fallback

    Version 1.2 [12th Nov] (beta)

    # Completely avoid "dumpsys" to backup app permissions
    # Now app data in tar.gz is unpacked in /data/data
    (probable fix to "Operation not permitted" issues in Google Pixel and some other devices)

    Version 1.1.1 [10th Nov] (beta)

    v1.1 (skipping upload)

    # [New] Backup verification before zipping
    # Major code refactoring and under-the-hood changes
    # CANCEL button now kills the backup process
    # Fix to "set_progress()" in recovery
    # Probable fix to "Failed to change locale for db"
    # Probable fix to missing updater-script
    # Probable fix to some "Operation not permitted" issues
    # Remove illegal characters from backup name
    # "Share scripts" option should now work properly.

    v1.1.1

    # Fix to a no-closing-quote issue in helper

    Version 1.0.5 [7th Nov] (beta)
    # Probable fix to "android.os.TransactionTooLargeException"
    # Better in app protection to prevent zips from not being made
    # App version is now written in logs
    # App now targets Oreo instead of Pie to potentially avoid some crashes
    # Fixed unexpected ( in restore process

    Version 1.0.4 [5th Nov] (beta)
    # Improvements to logging
    # Ignore tar socket warnings
    # New update-binary to prevent TWRP error 3
    # Should fix a null String issue while reading backup data

    Version 1.0.3 (beta)
    # Fixed log sharing
    # Share script file in log
    # Blacklist Android system from system app backup list (prevents tar socket error)

    Version 1.0 (beta)
    # Initial release
    32
    Hi @SayantanRC ,
    actually I'm a TWRP developer and i'm working on a lot of devices and recently a bit of users starting to tell me that they can't flash their migrate backups, so I found myself investigating a little and at the end i found the problem: in the migrate zips that migrate app creates, there are some commands that use busybox that actually is deprecated and not more used in TWRP. Can you please use generic commands or make a choice to use toybox instead of busybox?
    For example this command:

    run_program("/sbin/busybox", "umount", "/data");

    Can be replaced from this one:

    unmount("/data");

    Or another option is to include a fully busybox set of binaries in the migrate zips so they will be independent. If you need some help or explanations contact me.

    P.s. i already sent also a mail to your play store supprot email.