FORUMS
Remove All Ads from XDA

[Recovery][Kernel] Install TWRP, remove encryption, and get Magisk on the Visible R2

76 posts
Thanks Meter: 34
 
By FEGuy, Member on 1st August 2019, 05:11 AM
Post Reply Email Thread
Disclaimer: This guide and the tools included are provided as-is. Testing has been limited and nothing is guaranteed to work. I take no responsibility for lost data, bricked devices, etc. Continue at your own risk, and please read carefully.

Note that Magisk and TWRP can interfere with downloading and installing OTA updates. At this time the only updates are security patches, so it would be best to make sure all OTAs are installed before starting.

Mounting and decrypting the data partition is still hit-or-miss. If you haven't set up a passcode or password, TWRP should be able to mount it, and you still might be able to even if you have. Magisk will corrupt encrypted partitions and force a factory reset, and TWRP will not be able to help you back things up if Magisk has been installed before disabling encryption. You've been warned.

Guide: Installing TWRP and Magisk on the Visible R2
Tools:
prog_emmc_firehose_8917.mbn - firehose for flashing system partitions
Latest available TWRP zip - contains TWRP recovery image and the necessary files for flashing
fstab.qcom - for removing forced encryption
Patched boot image - for installing versions of Magisk newer than v16.7
Stock firmware - optional, but handy to have in case something goes wrong.

Windows users: QPST/QFIL and the QDLoad drivers (possibly the first set of ZTE drivers as well, but likely not necessary). Installing drivers in compatibility mode for XP or Windows 7 might help resolve issues.
Install adb if you don't already have it set up.

Linux users: qdl source code; the guide will cover compiling and installing it. You'll also want to install adb from your distro's repositories. There is a snap package for qdl, but it doesn't work for the purposes of this guide, so please don't use it.

Part 1: Installing TWRP
TWRP is an easy-to-use, touch-based custom recovery for Android devices, designed to make backups and installs simple and painless. Unfortunately, the installation of TWRP on the R2 is going to be a little less simple and painless; the device's bootloader isn't locked down, but it doesn't support fastboot commands, meaning we'll need to get our hands dirty to flash partitions.

Before starting, it's also worth noting that the current TWRP build cannot always decrypt encrypted data partitions, and the device is encrypted by default. This means that TWRP may not be able to mount or back up your data partition unless it's formatted and forced encryption is removed (see part 2).

To flash TWRP, we need to put the device into EDL mode and forcibly overwrite the existing partitions using an EDL tool and the firehose linked above, which unfortunately requires a PC of some sort. Enable USB debugging on your phone, then follow the steps for your operating system below.
For Windows:
1. Install the Qualcomm driver package. Before continuing, you will need to make sure Driver Signature Enforcement is disabled on your PC; QFIL may have issues communicating with the device if Windows blocks the driver.
2. Install the QPST package, then open QFIL from the Start menu.
3. Select 'Flat Build' under Select Build Type.
4. Extract the TWRP zip to an accessible directory (something like C:\TWRP works fine). Place the firehose.mbn in the same directory.
5. Press 'Browse' under Select Programmer, then navigate to the TWRP directory and select the firehose file.
6. Press 'Load XML'. Select rawprogram_recovery.xml, then patch0.xml
7. Connect the phone to your PC and make sure USB debugging is enabled. Make sure your PC is trusted by the phone for ADB commands, then send 'adb reboot edl'.
8. If everything went well, the phone should have rebooted with a blank screen, and QFIL should be asking you to select a port. Press Select Port, and then choose (hopefully) the only item available).
9. The 'Select a port' message should now read something like Qualcomm HS-USB QDLoader 9008 (COM#). IF it does, simply press download.
10. There should be some output in the status window, ending a few seconds later with a blue Download Succeeded message. If so, you're done, and can restart your phone. If there's a red error message, or the status window seems to be stuck on sending, double-check that your device is registered properly in Device Manager.
11. Once you're done flashing, you can reboot into TWRP by holding down the Vol Up button as the phone reboots.


For Linux:
1. Unzip the qdl zip into its own directory, then enter the directory.
2. Compiling qdl requires libraries that may not be installed. For Ubuntu users, you simply need to run ' sudo apt install libudev-dev libxml2-dev' to install them; people with other distros should know how to use their package manager and may have to find the packages under a similar name. If any other packages are required (errors compiling, etc.), let me know so I can update this.
3. Actually compiling should be as simple as opening a terminal in the working directory and running 'make && sudo make install'. If you get no errors, qdl should be installed and able to run from any location, and the compile process shouldn't take much time at all.
4. Before running qdl, ModemManager needs to be disabled, as it can interfere with accessing the phone in EDL mode. In Ubuntu (and most systemd-based distros) you can ensure that it's stopped by running ' sudo systemctl stop ModemMangager'. If you need the ModemManager service, make sure to restart it when you're done.
5. Extract the TWRP zip to an easily-accessed folder, like ~/twrp, and place the firehose mbn in the same folder.
6. Navigate to the TWRP folder and open a terminal there. Now would be a good time to plug in your phone and enter EDL with 'adb reboot edl'.
6a. If your phone was already connected in EDL mode before stopping ModemManager, you might need to reboot it and enter EDL again.
7. If you're in the same directory as the TWRP files, you should be able to start flashing by running 'sudo qdl prog_emmc_firehose_8917.mbn rawprogram_recovery.xml patch0.xml'.
8. If everything's good, you should see some output on your screen, and the phone should reboot momentarily. If the command finishes without output, ModemManager or something similar may have interfered. Make sure ModemManager is stopped, reboot your phone into EDL mode again, and try again. If it hangs at Waiting for EDL device, you're not running the program with admin privileges or your phone isn't in EDL mode.
9. Once the process has finished and your phone has rebooted, you should have TWRP installed. Boot into it by holding Vol Up while rebooting.


That's it! Verify that TWRP is working as expected by doing some test backups and restores, etc. Note that TWRP builds for this device are still sort of experimental; decrypting the data partition may not work for some users, and keeping data encrypted while trying to flash Magisk will not work - you will need to wipe and decrypt the device. If you're satisfied just with TWRP, there's not much else you need to do. If you want to run Magisk or decrypt your device for other reasons, keep reading.

Part 2: Removing Forced Encryption
By default, the device encrypts the data partition without input from the user. This is not ideal, and even if TWRP can decrypt and mount your data partition, from my experience Magisk's init process apparently gets things wrong and tries to write to data before its decrypted - whether that's actually the case or not, the fact of the matter is that Magisk has corrupted my data partition every time I've tried to install it while data was encrypted.

DO NOT flash any zip that removes dm-verity and force encryption - these modify the boot and recovery partitions, and the stock kernel doesn't like the way they do it. If you've flashed the patched boot.img first then you'll at least be able to boot to your system partition afterwards, but it'll still nuke recovery even with a custom kernel installed on it. If you're still using the stock boot.img, you'll get boot loops and won't be able to access recovery; your only choice is to enter diagnostic mode, and reflashing anything from that point is a gigantic pain in the ass, so please, just don't. If you feel the need to do it regardless, please please please reflash your boot and recovery images before rebooting.

Counter-intuitively, the quickest and most painless way to stop the device from force-encrypting itself is to modify the fstab on the vendor partition - the boot partition has no fstab files, and the ones in the TWRP recovery image are already set to make encryption optional. Place the fstab.qcom on the phone's microSD card (internal storage will work if you don't have one, as long as TWRP can access your internal storage; if not, use adb push/pull rather than the below commands), boot into TWRP, and adb shell into your device by USB; if done while in TWRP, you should have root permissions. Run the following commands:
Code:
mount /vendor
cp /vendor/etc/fstab.qcom /external_sd/fstab.qcom.bak
cp external_sd/fstab.qcom /vendor/etc/fstab.qcom
chmod 644 /vendor/etc/fstab.qcom
umount vendor
.
That's the simple part - you've replaced the fstab on the vendor partition with one that tells the device that encryption is optional, and backed up the original to your microSD as fstab.qcom.bak, just in case. Now you need to actually remove the existing encryption, which will wipe all the user data from the device. If that hasn't scared you out of continuing, read on.

While we're still in TWRP, we need to go to the Wipe menu. From there, we need to Format Data. It'll give you warnings in a scarier color than I did, but you'll have to type yes and continue regardless. Once that's done, hit the back button a couple times before rebooting. Do a standard factory reset, then reboot.

From there, it may take a couple tries for the device to boot normally as it reformats and repopulates the data partition. If you find yourself stuck on a black screen, reboot again, and if it keeps happening, boot into TWRP and then reboot to System from the reboot menu. Eventually, you should be able to get back to the device setup screen, and depending on your security settings beforehand, you may be prompted to enter your PIN, password, or Google account information.

From here, you should be able to mount, backup, and restore your data partition in TWRP without any issues. If you want to go further and root your device, keep reading for the Magisk guide.


Part 3: Any Magisk You'd Like
Note: Magisk and encrypted data partitions do not play nicely together. Follow through Part 2 first, or you'll have headaches.

ZTE's kernel has a custom SELinux plugin called policyproc, which in short doesn't play nice with a lot of things that modify what happens during the kernel's startup sequence. Unfortunately, modifying the startup sequence is what Magisk does best, and versions of Magisk past v16.7 don't play well with the stock kernel.

If you're fine with v16.7, you can download the zip from its Github release page and flash it through TWRP. However, it doesn't support current versions of Magisk Manager or the uninstaller zip, and naturally it doesn't have all the features of newer versions.

This is where the patched boot image comes in. Basically, it's the stock kernel, rebuilt with policyproc disabled, stuffed into the stock boot.img and replacing the original kernel. It's not entirely perfect as ZTE deliberately left out bits of the source, but even working around that, the kernel seems to run fine after nearly a week of testing on my own device. Still, if you run into any issues that don't show up with the stock kernel, please let me know.

So. you can take that patched boot image and install newer versions of Magisk whichever way you'd like. The default method would be to use TWRP to flash the patched boot image, then flash the latest Magisk zip (v19.3 at the time of writing). Alternatively, you could put the patched boot.img on your microSD or internal storage, install the Magisk Manager app, use it to patch Magisk into the patched boot image, then use TWRP to flash the patched-patched-image to the boot partition. Either way should work fine, and honestly I'm hoping if you've made it this far you don't need further instruction on how to get Magisk installed.

Note that on the first reboot after installing Magisk, the phone may reboot again before loading the system normally. I assume this is standard practice for initializing Magisk, but honestly I have no idea.

With that, you've done just about everything you came here to do. If you still feel like tweaking and possibly bricking your phone, however...


Part 4: Going Even Further Beyond - Project Treble and You
Google requires that all Android phones releasing with Android Oreo or newer must support the Treble framework: generally speaking, anything that's 'stock' Android lives on the system partition, and anything manufacturer or carrier specific exists on the vendor partition. This has allowed for developers to create generic system images, or GSI, consisting of particular Android system partitions that should run on a variety of Treble-enabled devices. And wouldn't you know it, the R2 launched with Oreo, and you've got a couple handy methods of writing system images to your device, through TWRP and EDL flashers.

So, does that mean the R2 supports these GSIs? Not really! Honestly, it's more that it's up to curious people to find out. I've been able to get phhussons's Oreo image working, but that's a bit boring on a device that supports Oreo out of the box. A couple more highly modified Pie builds failed to boot properly, either hanging on their splash screen or crashing the display manager before getting through setup.

There's also the fact that they're generally only distributed as sparse system.img files, something I had trouble flashing with QFIL, and up until about 18 hours before this guide went live, the only TWRP build I had for the R2 could only write images to the boot and recovery partitions, not system, so writing them meant I had to decompress them to a standard EXT4 image and then flash them with QFIL, which wasn't a lot of fun to mess with. In short, testing's been pretty limited, but this build of TWRP should make it much easier for anyone to flash whatever image they'd like. If you feel like being a guinea pig, we'd all love to hear the results! If you want to test, you should look for A-only, ARM64 images.


Questions and Issues
My phone is bootlooping after leaving recovery!
If you haven't messed with system partitions in a way that might cause a bootloop, reboot to recovery and try to boot to system from there. If it continues, reflash your boot image (and Magisk afterwards, if necessary, removing the extra Manager app that might pop up in your app tray if you've got Manager disguised in its settings).

I can't access ADB or MTP from recovery.
It happens sometimes, especially in Windows; just toggle MTP on/off from inside TWRP, or unplug and replug your USB cable.

My phone says I need a factory reset, but TWRP just reboots.
This generally happens if something messed with the encrypted data partition. Do not tap the factory reset button on your phone; instead, power off entirely and manually reboot into TWRP by hilding Vol Up while booting. Perform a factory reset (and format data if necessary) from there, then reboot.

(to be populated)


Credits
@deadman96385 - for uploading the tools that made this possible and the stock ROMs that saved my ass more than a few times
@famewolf - for feedback on what did/didn't work in the last guide
@asderdd - for his Axon 7 kernel work which I shamelessly copied to get Magisk working
Anyone who's contributed to Magisk or TWRP over the years, and the maintainers of the moto e5+ device tree for TWRP since I used most of their branch.

Sources
Kernel source, specifically the tree used for the Magisk patch
TWRP device tree
The Following 6 Users Say Thank You to FEGuy For This Useful Post: [ View ] Gift FEGuy Ad-Free
 
 
1st August 2019, 05:46 AM |#2  
famewolf's Avatar
Senior Member
Thanks Meter: 1,307
 
Donate to Me
More
@FEGuy

Attempted to flash the new twrp via zip....acted like it worked but when I when to install section only boot and recovery were options....so I installed the twrp image itself to recovery....rebooted back into recovery and got a black screen. Additional attempts to go into recovery did same thing. I'll attempt to use the flashify app under factory rom to restore previous copy of twrp. It should work as I had magisk going...I think.
1st August 2019, 06:11 AM |#3  
OP Member
Flag Marquette, MI
Thanks Meter: 34
 
More
That is... not really reassuring. I've suspected the zip flash might not actually work, but I double-checked that TWRP build on my device first.

What's weird is that that's also the only TWRP image I've gotten to boot without dragging it through an AVB signing process. Let me sign that image and reupload it, see if it works any better.
1st August 2019, 06:21 AM |#4  
famewolf's Avatar
Senior Member
Thanks Meter: 1,307
 
Donate to Me
More
Quote:
Originally Posted by FEGuy

That is... not really reassuring. I've suspected the zip flash might not actually work, but I double-checked that TWRP build on my device first.

What's weird is that that's also the only TWRP image I've gotten to boot without dragging it through an AVB signing process. Let me sign that image and reupload it, see if it works any better.

Always possible I hosed something....I grabbed the img of the previously good twrp and flashed it via flashify app...said it worked but when booted to recovery got same black screen. I'm about to experiment with the usb drivers you mentioned vs the ZTE USB ones in the op...they conflict with each other so uninstalling old ones.
1st August 2019, 06:38 AM |#5  
OP Member
Flag Marquette, MI
Thanks Meter: 34
 
More
If the ZTE and QDLoad drivers conflict, I'd probably say keep the QDLoad. I honestly don't remember if I ever installed any ZTE drivers other than the ones that can be served from the phone by USB.

Pretty sure flashify is just broken; I tried it the other day with the same results after flashing a test build that didn't work at all.

I just redownloaded the TWRP from the original link and it worked fine, and there shouldn't be any files or signature tied to my own device, so I don't really know why it wouldn't work on other phones. I've got another one coming in the mail for actual use on Visible, but it won't be here to test on until tomorrow.

In the meantime, try this TWRP. No different other than letting AIK give it an AVB signature, but some of my test images wouldn't boot otherwise. If it works I'll update the zip in the first post.
1st August 2019, 07:01 AM |#6  
famewolf's Avatar
Senior Member
Thanks Meter: 1,307
 
Donate to Me
More
Quote:
Originally Posted by FEGuy

If the ZTE and QDLoad drivers conflict, I'd probably say keep the QDLoad. I honestly don't remember if I ever installed any ZTE drivers other than the ones that can be served from the phone by USB.

Pretty sure flashify is just broken; I tried it the other day with the same results after flashing a test build that didn't work at all.

I just redownloaded the TWRP from the original link and it worked fine, and there shouldn't be any files or signature tied to my own device, so I don't really know why it wouldn't work on other phones. I've got another one coming in the mail for actual use on Visible, but it won't be here to test on until tomorrow.

In the meantime, try this TWRP. No different other than letting AIK give it an AVB signature, but some of my test images wouldn't boot otherwise. If it works I'll update the zip in the first post.

If I su from adb and do a dd if=twrp-signed.img of=xxxxxxxxxxx can I overwrite it manually from shell and if so do you know the path I should use to get it to our recovery?

If flashify is broken, twrp is broken and I'm unable to use qdl nor qfil I'm kinda screwed (for the moment). The qdload drivers did not show the device on a com port when in edl mode...the zte ones do but then again I can't write anything. Any ideas what the heck is up with the linux qdl? Even with --debug it says nothing.

---------- Post added at 02:01 AM ---------- Previous post was at 01:48 AM ----------

Quote:
Originally Posted by famewolf

If I su from adb and do a dd if=twrp-signed.img of=xxxxxxxxxxx can I overwrite it manually from shell and if so do you know the path I should use to get it to our recovery?

If flashify is broken, twrp is broken and I'm unable to use qdl nor qfil I'm kinda screwed (for the moment). The qdload drivers did not show the device on a com port when in edl mode...the zte ones do but then again I can't write anything. Any ideas what the heck is up with the linux qdl? Even with --debug it says nothing.

I tried dd if=twrp-3.3.1-1-z5151v.img of=/dev/block/bootdevice/by-name/recovery and did same with new signed one. I still get black screen when recovery "loads". At this point I'm going to have to get something figured out with qdl or qfil.
1st August 2019, 07:11 AM |#7  
OP Member
Flag Marquette, MI
Thanks Meter: 34
 
More
Yeah, I wasn't sure if dd was going to work or not.

Try the QDLoad drivers again in Windows, make sure that the device is using them. If it's not showing up as a COM port in device manager, force it to use the proper Qualcomm driver, which on my system installed to C:\Windows\system32\qcusbcer.sys - I don't know if that installed with the QDLoad drivers or with QPST/QFIL itself.

As for qdl, I don't know; it's been hassle-free for me, for the most part. If you're sure you're in EDL mode and not diagnostics or something, an output from lsusb might help diagnose the issue. If the debug flag isn't providing useful info, might as well try it without. I don't know if there's services other than modemmanager that might interfere with qdl, it could depend on distro.

EDIT: I'll also preemptively warn against trying the TWRP app to flash; it'll do about as much good as flashify.
1st August 2019, 07:28 AM |#8  
famewolf's Avatar
Senior Member
Thanks Meter: 1,307
 
Donate to Me
More
Will tackle it "later" in the morning. Thanks for the quick followup. I might see more clearly in morning when not so frustrated with it. I swear I haven't had this many issues with getting twrp installed in a long time.


*update* Ok so I lied....I completely recompiled qdl from https://github.com/andersson/qdl and it's working as expected..I used it to flash the original twrp. I then used that twrp to flash the twrp-signed.img and rebooted back into it with no issues.

I also flashed your patched boot and it got as far as the blue visible screen after the zte one then hung. I reverted to my nandroid copy of boot.
1st August 2019, 08:19 AM |#9  
famewolf's Avatar
Senior Member
Thanks Meter: 1,307
 
Donate to Me
More
The Treble pie loaded ok...I did do a factory reset so data would not conflict..it did do one reboot before loading normally. See screenshots.


The First 4 are from when it was initially installed. The later photo's I had added gapps, a few apps, snapped a picture to use as wallpaper and various other minor things. I found it to be faster under treble pie than it was with factory oreo. I haven't found anything that didn't work yet on it.

Launcher: Evie Launcher
Attached Thumbnails
Click image for larger version

Name:	Screenshot_20190801-031010.png
Views:	130
Size:	90.6 KB
ID:	4800208   Click image for larger version

Name:	Screenshot_20190801-031020.jpg
Views:	132
Size:	44.7 KB
ID:	4800209   Click image for larger version

Name:	Screenshot_20190801-031028.png
Views:	130
Size:	185.6 KB
ID:	4800210   Click image for larger version

Name:	Screenshot_20190801-031047.png
Views:	126
Size:	102.0 KB
ID:	4800211   Click image for larger version

Name:	Screenshot_20190801-122436.jpg
Views:	112
Size:	82.2 KB
ID:	4800368   Click image for larger version

Name:	Screenshot_20190801-122452.jpg
Views:	108
Size:	75.8 KB
ID:	4800369   Click image for larger version

Name:	Screenshot_20190801-122500.png
Views:	108
Size:	136.9 KB
ID:	4800370   Click image for larger version

Name:	Screenshot_20190801-122640.png
Views:	106
Size:	64.7 KB
ID:	4800371   Click image for larger version

Name:	Screenshot_20190801-122705.png
Views:	106
Size:	52.7 KB
ID:	4800372   Click image for larger version

Name:	Screenshot_20190801-122726.jpg
Views:	102
Size:	81.3 KB
ID:	4800373   Click image for larger version

Name:	Screenshot_20190801-122747.jpg
Views:	96
Size:	50.2 KB
ID:	4800374   Click image for larger version

Name:	Screenshot_20190801-124613.png
Views:	92
Size:	84.7 KB
ID:	4800377  
1st August 2019, 08:31 AM |#10  
OP Member
Flag Marquette, MI
Thanks Meter: 34
 
More
Quote:
Originally Posted by famewolf

I also flashed your patched boot and it got as far as the blue visible screen after the zte one then hung. I reverted to my nandroid copy of boot.

Did you still have Magisk installed at the time? I'd have recommended uninstalling Magisk using the uninstaller zip from the 16.7 release before replacing the boot image; if there's still Magisk data in /data then booting with a non-Magisk boot image is probably going to cause issues.

In the meantime, I guess I'll update the TWRP zip with the signed recovery image, just in case.
1st August 2019, 02:06 PM |#11  
famewolf's Avatar
Senior Member
Thanks Meter: 1,307
 
Donate to Me
More
Quote:
Originally Posted by FEGuy

Did you still have Magisk installed at the time? I'd have recommended uninstalling Magisk using the uninstaller zip from the 16.7 release before replacing the boot image; if there's still Magisk data in /data then booting with a non-Magisk boot image is probably going to cause issues.

In the meantime, I guess I'll update the TWRP zip with the signed recovery image, just in case.

I tried the current installer which failed. I didn't locate the 16.7 uninstaller.

*update* I did finally locate the 16.7 Magisk Uninstaller.
Attached Files
File Type: zip Magisk-uninstaller-20180719.zip - [Click for QR Code] (2.25 MB, 68 views)
Post Reply Subscribe to Thread

Tags
zte-visible-r2

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes