Remove All Ads from XDA

[2019.9.19] Magisk v19.4 - Root & Universal Systemless Interface [Android 4.2+]

1,839 posts
Thanks Meter: 57,369
By topjohnwu, Recognized Developer / Recognized Contributor on 3rd October 2016, 06:00 PM
Thread Closed Email Thread
1st June 2017, 02:19 PM |#21  
OP Recognized Developer / Recognized Contributor
Flag Taipei
Thanks Meter: 57,369
Donate to Me
Why is Magisk Manager not available on Play Store? Is development abandoned?
This is what I get from Google this morning:

Magisk Manager, com.topjohnwu.magisk, has been suspended and removed from Google Play as a policy strike because it violates the malicious behavior policy.

What is the so-called "malicious behavior"? From what I've suspect, viewing the definition of malicious behavior, most likely I violated the two following policies:

  • Apps that introduce or exploit security vulnerabilities.
  • Apps or SDKs that download executable code, such as dex files or native code, from a source other than Google Play.

For the first policy: Magisk bypasses Google's strict compatibility check - the CTS check on tampered devices (SafetyNet checks CTS status). CTS is what Google judge whether a manufacturer can ship a device with its Google services, so Google is definitely really serious about this issue. Also, Magisk roots your device, patches tons of SELinux policies (all rooting method do) etc, which is also an obvious security breach.

However, I doubt this was the main reason, since many superuser management apps are also on the Play Store. The main reason should be the other one.
The second rule I listed can be translated to: you cannot have anything "market-like" to let users download and run code on your device. Apparently, Magisk's Online Repo is a complete violation against this rule.

Now I have two choices: Remove the online repo from Magisk Manager, and re-release a NEW APP on the store (yes, once your app is pulled down, the package name and app name is permanently banned).
The other way is to simply just distribute the app through places like XDA and third-party markets (just like Xposed Installer).
I prefer the second decision, because I can still use the same package name, also I wouldn't need to remove the online repo feature, which is one of the most precious thing for a development community like XDA. What I really lost is the $25 dollars for Play Store registration lol.

Development is definitely NOT suspended in any way, in fact, I had significant progress lately.
There are still some bugs not sorted out, and I need some feedback from the users, so I decide to start a new thread for public beta testing!
Expect the new thread to be live very soon, but I still need to do some small adjustments to deal with the unfortunate Play Store situation....

So the conclusion is: Yes, Magisk Manager is pulled from Play Store due to policy violation; and no, this is not a sign for the end of development.
In fact, I think Magisk is undergoing the most active development since release!
The Following 1,117 Users Say Thank You to topjohnwu For This Useful Post: [ View ]
7th June 2017, 10:33 PM |#22  
OP Recognized Developer / Recognized Contributor
Flag Taipei
Thanks Meter: 57,369
Donate to Me
2017.6.8 [BETA] Magisk v13.0
Due to the fact that this release is nearly completely rewritten from scratch compared to the last release, I decided that it will be a better idea to release it as a beta, since many things might not work properly due to the massive changes.
On the other hand, many previous bugs might be fixed due to complete different structure.

Unified Binary
As I previously mentioned in an update post, this release comes with the unified Magisk binary. The binary itself can be used as resetprop, su, magiskpolicy, magiskhide. Along with this change is that all operations are running in the same "unified daemon", including all boot operations, selinux live patches, magic mount etc. Even though this only covers one line in the changelog, it actually takes a lot of effort to redesign a lot of details.
Magisk only requires a single binary (magisk) to be injected into /sbin, an rc script defining all entrypoints, sepolicy minimally patched in order to start magisk daemon to fully work on a device. The ext4 image creation, resize, merging etc. are all handled automatically by the daemon.

Magisk Manual Injection
Due to many changes in this release, it is possible to patch a boot image on the Android device without root.
The only requirement is the boot image itself (it is not possible to dump the boot image from the device without root, you will need the boot image from somewhere else), the ability to flash a boot image (most likely an unlocked bootloader), and a few adb commands.
This doesn't seem like a big deal, but for devices with no custom recovery, this will be a good way to start rooting and do further modification on your device (flashing zips through FlashFire, adding modules by Magisk Manager etc.)
I will post the simple instructions soon, and this boot patch without root feature is planned to be added into Magisk Manager in the future.

MagiskSU Massive Improvements
I spent some time to improve the included root a lot. Added multiuser support, added re-authentication support, and tons of bug fixes.
Another point worth mentioning is the new timeout queue for root requests. In this release, root requests from the same requester will use a cached policy and suppress logging if called within 2-3 seconds. This will prevent heavy performance hit from poorly programmed root applications, which calls su for excessive amount of times within a few seconds.

(Addition in 2017.6.9)
New namespace mode options are added in build (0b4baad). In versions before this build, all root sessions run in the same global namespace. Starting from this build, the default mode will inherit the requester's namespace, so an application will internally share the namespace for both privileged and unprivileged processes, but do not share across different applications. Users can choose to use the global namespace, or opt in a stricter "isolated mode", which each root session will have its own separated, isolated namespace.

Temporarily Dropping SuperSU Compatibility
This is not a move against SuperSU. In previous versions, in order to support merging Magisk modifications into SuperSU without breaking stuffs requires TONS of handling in the scripts, and I don't think supporting an already patched boot image worth the effort. I plan to release a su.d Magisk alternative for SuperSU users who wants to use resetprop and magic mount features.

Documentations, module template updates, manual injection instructions will be updated in the few days.
The Following 803 Users Say Thank You to topjohnwu For This Useful Post: [ View ]
10th July 2017, 10:15 PM |#23  
OP Recognized Developer / Recognized Contributor
Flag Taipei
Thanks Meter: 57,369
Donate to Me
2017.7.11 Magisk v13.1
Wow, what a journey! The last stable release is in April, nearly 3 months ago! As stated before, even though the functionality is the same as previous "script based" releases (anything prior to v13), the underlying mechanisms are actually significantly different from all aspects. You can consider this as a complete different project from previous versions lol. Here I will just do some announcements instead of going through all changes. The changelog now shows proper v12.0 -> v13.1 changes instead of changes for each beta iteration. Those interested in the actual changes can take a look there.

Completely Drop Support for Older Versions
Due to the massive difference between v12.0 and v13.X+, older "script based" versions were and will be never, ever put into consideration in development. There will be a lot more things that is no longer backwards compatible, and staying on an old release will let you miss out a tons of fixes and improvements, so please be sure to upgrade to the latest version.

New Magisk Module Template v4
Finally, the module template received an update! The new upgrade brings proper Android O handling, and many fixes for flashing in all kinds of environment. New changes comes with new things that need to be noted. You will find out that the commands in the flash script are now greatly reduced, with only the essentials and function calls remain. The script now relies on the main binary /data/magisk/magisk, and the shell script function collection /data/magisk/, both come along with a proper Magisk v13.1+ installation. These changes are due to the fact that a lot of the ext4 image handling are now offloaded to the magisk binary, since there is no real way to properly handle these operations aside from using my own written tools, one factor being the removal of the bundled busybox. Another major advantage is that since the actual logic and operations are not bundled in the template itself but rather included in Magisk, once a bug or improvement is found, the changes can simply be pushed through a Magisk update instead of updating the template, which requires module developers to migrate to new code. It is similar concept to what "shared libraries" are doing.
However, due to these changes, Magisk Modules are no longer flashable in custom recoveries without /data access. You have to either flash in proper configured custom recovery with /data access, or flash within Magisk Manager (the scripts and merge handling are greatly improved compared to old versions, worth trying!)
The next release (not this one) of Magisk Manager will filter out all modules on the repo with template version lower than v4, developers please find some time to upgrade your modules, I believe it will only take you a few minutes!

Module Repo Submission
Many could have questioned why I haven't accepted a single module into the repo for quite a while, not because I don't care about the community (I gave up Play Store for this, how come will I abandon it), the reason is because I have been working hard to finalize Magisk, which only after then I can decide the final format of the template. With the new Magisk comes to stable, and a lot of thoughts have been into, things should rarely change, at least not significantly, and as a result module submissions will start get accepted once they are adapted to template v4.
This summer vacation I have an internship, so things are different compared to last summer when Magisk was born from my endless and sleepless development - I have proper work to do now lol. Also due to the fact that it's going to be my last year in university, I need to start preparing for applying graduate school; the free time will be very limited. I will focus more on maintaining the online repo, and slow down the development of Magisk, which means submission should be addressed in a timely manner.

Play Store Device Certification
There is a new thing within Play Store called "Device Certification" (you can check through settings). Currently I cannot find how it works, it always shows "Certified" on my Nexus 5X on O preview and Galaxy Note 10.1 2014 running LineageOS 14.1, however no matter what I do, my HTC U11 always shows "Uncertified". Currently this isn't a big deal though, I won't spend time investigate this.

Some Miscellaneous Stuff
Some might have noticed, the scripts under /magisk/.core/post-fs-data.d and /magisk/.core/service.d requires execution permissions to be executed, this is done intentionally as some developers rely on the file permissions to toggle scripts. and in modules, on the other hand, doesn't have this requirement. These scripts should be toggled along with the module, not based on the permission of the file itself.

On Android O, setting system props through property_service (the setprop command, and resetprop by default) in the post-fs-data stage will cause the boot progress to halt - it will wait till the timeout set in Magisk init scripts is over, doing nothing in the meantime. This will cause super long boot times and prevents Magisk to work properly. In previous Android O previews, this even prevents the device to boot up.
Important: I HIGHLY suggest all developers/users to run their scripts in service mode UNLESS necessary. If you had to set props in post-fs-data (build related ro props for example), please consider using system.prop in modules as Magisk will handle it properly. If you really need to set props through post-fs-data scripts, call resetprop with the flag "-n" enabled, this will bypass the property_service.

With the introduction of mount namespace separation, some root developers contacted me for a request for "master-mount" option. This is added to MagiskSU, and calling su with this flag will give you the global mount namespace regardless of user configuration within Magisk Manager. Useful for root developers which need to global mount namespace for managing mount points.

Final Words
The documentation is super outdated now, and I have previously promised for an update but it never happened lol.
I already start working on it, too many has changed and some stuffs has changed seriously. A major new release should come with proper documentation
I will post another update once the documentations are done, I will point out some of the most important ones.
The Following 936 Users Say Thank You to topjohnwu For This Useful Post: [ View ]
13th July 2017, 08:10 PM |#24  
OP Recognized Developer / Recognized Contributor
Flag Taipei
Thanks Meter: 57,369
Donate to Me
2017.7.14 Magisk v13.2
Here comes the release to fix many fairly small but critical issues in the last release

MagiskPolicy Update
Thanks to @jenslody for finding this bug! This is the reason why many cannot make Magisk v13.1 run on your device, since selinux patching is a critical part in both Magisk's boot patch and boot sequence. Most victims seems to run on Marshmallow. If your device wasn't running v13.1, this release worth a shot

/sbin Re-link Fixes
One of the most important part in MagiskHide is the /sbin re-linking part. In previous versions there is a bug where the selinux context was set to a non-target folder, which casues some issues. Also, along with the bug fix, I also adjusted the context to match other files in the ramdisk, so that it might also eliminate a lot of weird issues (phone feature breaking non-related to Magisk) if MagiskHide is enabled. For one that I know is that HTC U11 users suffering the USB-C headphone jack adaptor malfunctioning should now be fixed.

Please Migrate Modules to v4!!
Currently, there is only few modules on the repo that has updated to the new template format. This release still WILL NOT block out v3 modules, but this will happen most likely when the next verison comes out. For the many module submissions on Github, please also update your module to the new template. Many thanks!!
The Following 592 Users Say Thank You to topjohnwu For This Useful Post: [ View ]
18th July 2017, 07:28 PM |#25  
OP Recognized Developer / Recognized Contributor
Flag Taipei
Thanks Meter: 57,369
Donate to Me
2017.7.18 Magisk v13.3
Google really doesn't want me to rest, let's just jump right into it, shall we?

This update is pushed out earlier than I planned to do so, but apparently users cannot wait anytime longer for the SafetyNet fix lol. This time, the additional detections are fairly easy to figure out, some even cracked it before I even know the issue exists.
This particular change is still within my expectations; in fact, being the creator of Magisk, I personally have TONS of other ways to detect Magisk, apart from "package name" detection which I currently do not have an optimal solution, I can hide all the other methods (note: I'm no security expert, this is only MY point of view, a university student who can't stop coding). I won't go on and implement all the hiding techniques and put all my cards on the table though, Google will slowly add more and more detections, and I'll just update Magisk each time something breaks. I believe there is still a long way till I run out of ideas

Google is targeting system props this time, resetprop FTW! The init process will record all start up services and set a property for each to indicate the state of the service. Well, we'll just parse through all props, and remove those which Google doesn't like! Something worth noting though, previously deleting props only happens in the memory, so the props disappears immediately, but persist props will actually stick for the next reboot. Resetprop is updated to support completely eradicating persist props when using its delete command, and the "-n" flag can give you the option to only modify the data structure within memory without actually touching anything.
The uninstaller is also updated to remove all persist prop it left behind.

Magisk Manager
The big thing here is actually Magisk Manager! I have massively rewritten a lot of portions of the code, mostly under-the-hood changes. However, there is a feature I always wanted.
You can already enjoy the new flashing log screen if you upgrade Magisk Manager before upgrading Magisk through the application!
A video is always better than a million of words, check it out with your own eyes

The Following 781 Users Say Thank You to topjohnwu For This Useful Post: [ View ]
19th July 2017, 08:51 PM |#26  
OP Recognized Developer / Recognized Contributor
Flag Taipei
Thanks Meter: 57,369
Donate to Me
2017.7.20 Magisk Manager v5.1.1
Here comes the update of Magisk Manager to fix the notorious "app cannot start" issue. Fun fact: this issue is not even my fault!

According Posix Standards, a line in Unix is defined as a sequence of characters following a terminating <newline> character. So what this means is that if the last line of the file does not end with a newline, technically speaking the text file is not "ended". Magisk Manager didn't handle this situation, so it simply hangs there while parsing an improperly formated module.prop file, since it cannot find the end of the file. Magisk Manager will now handle this properly, so things should work just fine! in the OP is re-uploaded with Magisk Manager v5.1.1 bundled, so users flashing a newly downloaded zip will not suffer from the bugs in the older versions. I'm constantly improving the Magisk Manager since the app code is a bit crappy (not an expert in Android Application development...), so each Magisk Manager update should come with many improvements and updates.
The Following 644 Users Say Thank You to topjohnwu For This Useful Post: [ View ]
16th August 2017, 09:28 PM |#27  
OP Recognized Developer / Recognized Contributor
Flag Taipei
Thanks Meter: 57,369
Donate to Me
Several Announcements
It has been really busy for me lately, having a daytime job as an intern isn't that easy

Magisk Module Template v5 Removed
As stated in the previous beta release notes, the v5 module template is considered as a developer preview. I haven't fully tested the template yet, and apparently there are some critical bugs, so I temporarily removed v5 template from Github.

I promised a documentation for quite a while, and it finally happened! The original Wiki thread here on XDA is discontinued, all new documentations are hosted on Github.
The links to the docs are in the OP, or here

Install Magisk With ADB: No Custom Recovery Needed!
A lot of work were already done to make patching boot images in a non-root shell work properly, even the Magisk binary itself has added some functionality to support it. However, it wasn't fully tested, and the instructions never came out. Since the busybox environment started from v13.5, a consistent shell was established, so I took a bit more effort to finalize the way to manually inject magisk into a boot image. The method is added to the Installation section in the OP, people interested can check them out!
Note: This new installation method only works on the beta version v13.6, go to the beta thread for more info
The Following 284 Users Say Thank You to topjohnwu For This Useful Post: [ View ]
21st August 2017, 07:40 PM |#28  
OP Recognized Developer / Recognized Contributor
Flag Taipei
Thanks Meter: 57,369
Donate to Me
More Announcements
These few months has been really busy (I think I've said this for like 1000 times...), so I had little time to do proper Magisk maintenance. However, today is Solar Eclipse day (for those in the US), and also the Android O day, I guess I'll announce "something" to celebrate it!

Hiding Magisk Manager
Despite the limited free time, I managed to squeeze out this highly anticipated feature into Magisk Manager, thinking the implementation would be interesting but not too complicated.
Here is a demo video in action:
The method is far from perfect, but good enough to hide from many existing detection methods. As you can see in the video, after hiding Magisk Manager, it will automatically install a new app called "Unhide Magisk Manager". The cool stuff here is that the new app is given a new random package name and signed with test keys on-the-fly each time you hide Magisk Manager. This will prevent detection methods from detecting "Unhide Magisk Manager"'s' package name, a bit more "future-proof".
There are still some other stuffs I'd like to polish, expect this new feature to be available in the next beta release.

Pixel (XL) Support
People following me on Twitter might have known that I recently got my hands on a Pixel XL. After some brief research and investigation, the situation is not as complicated as I initially thought, so official Pixel support should be possible and pretty clean. However, it still requires quite a lot of adjustments, I wouldn't work on it until I have more free time, and also I would focus on pushing out a new stable release before I dive full in Pixel devices. In the meanwhile, Pixel (XL) users can use the unofficial builds if you want.

Unofficial Sites
I'm aware of some sites (such as https://******************) using the Magisk name to distribute Magisk. To be clear, all binary distribution are and will be hosted on XDA or my Github account, I do not own or be affiliated with these websites. I'm actually fine with websites like this using my project to gain ad revenue lol, but I still recommend users to download and seek for support from official sources, or even build Magisk yourself so it won't include any malware.
The Following 419 Users Say Thank You to topjohnwu For This Useful Post: [ View ]
6th September 2017, 05:09 PM |#29  
OP Recognized Developer / Recognized Contributor
Flag Taipei
Thanks Meter: 57,369
Donate to Me
2017.9.6 Magisk v14.0
Finally got some free time to work on development! Before adding the Pixel (XL) mess into the mix, I figured I would focus on stabilizing the current version, and make installing easier and more robust.
There are tons of under-the-hood changes compared to the last stable version, here I'll summarize the changes (include those in the betas) all in one post.

Busybox is Back
I have constructed a setup to easily build busybox for Android, and it has been included into Magisk. Also, a cleverly crafted update-binary, a shell script that has busybox embedded and can extract binaries matching the architecture on-the-fly, is used as a basis for installing. That busybox binary will also be used in various places such as all boot scripts and Magisk Manager, but will not be exposed externally. In order to install busybox systemless-ly, please install the Busybox module from @osm0sis, which is available on the module repo.

Samsung Stock Kernel Workarounds
Tons of changes has been done to overcome some weird issues on Samsung devices, including exec and fork issues. All functionality are thoroughly tested to work on both Samsung and non-Samsung devices.

Hiding Magisk Manager
This new feature has been added to Magisk Manager. The approach is pretty naive, as it can be still detected in some way, but it should work in many cases. When select to hide Magisk Manager, it will randomly generate a new package name (so it can't be blacklisted) and install a new app called "Unhide Magisk Manager" to your device, then it will eventually hide itself. While Magisk Manager is hidden, root will work with apps that were granted before (the database is still there), but no new requests can work since the application itself is hidden and all components are disabled. To unhide Magisk Manager, simply just open "Unhide Magisk Manager", Magisk Manager will immediately be unhidden and started up, while the unhide app will automatically remove itself.

New Installation Method
Nearly a month ago, I introduced a way to install Magisk through ADB. This is nice for devices with no custom recovery support, or for people like me who wants to preserve stock recovery to apply OTAs. To make things even easier, I have added this feature natively into Magisk Manager. In Magisk Manager you can provide your stock boot image (in both raw image format or tar-ed up ODIN flashable format), the app will then patch the provided image. Other required files and scripts are extracted within the app's data, and will be picked up automatically while booting up once with a magisk patched boot image. Check the new instructions in the OP for more info.
Once your device has Magisk installed, you can install Magisk modules through Magisk Manager without custom recoveries. The powerful systemless interface means that you can literally do anything to your device - of course systemless-ly!

New Magisk Manager Features
Apart from the new installation method, some other new features are also added. You can now restore your stock boot image by selecting Uninstall > Restore Stock Boot. Personally I find it very useful: after restoring to stock boot image (without reboot), I can directly apply OTAs on my device. Of course, you need to have stock recovery installed (thanks to the new installation method, I refrain from installing custom recoveries), or if supported you can use Chainfire's FlashFire to apply the OTA. After applying the OTA, you have to re-root your device again, or if using FlashFire you can just choose to flash Magisk along with the OTA.

Another feature is the introduction of update channels. You can now opt-in to receive updates from the beta channel through Magisk Manager.

New Magisk Module Template - New Versioning Scheme
Modules Developers please pay attention to this part!
A new template has been pushed to the repo. Starting from this version, the template version (the template entry in module.prop), is now the minimum supported Magisk version code. So for this particular release, the template version will be "1400", which is the version code of Magisk v14.0 (1400).
The updated template comes with tons improvements here and there, so repo developers please migrate to the new template ASAP.
The reason why there is a minimum required Magisk version has been mentioned before: it is due to the fact that most installation logic is NOT in the modules, they are shipped in a universal script with Magisk.

Also, starting from v4, all previous versions will be preserved on the Github repo in the branch name in respect to its version.

The default branch, which should be the latest one, will only have two commits:
"Initial Commit" --> "Magisk Module Template <version_num>"
All changes are squashed into one single commit. It will be much cleaner for new module developers to start from

While other older branches will have three:
"Initial Commit" --> "Magisk Module Template <old_ver_num>" --> "Template <old_ver_num> -> <next_ver_num>"
The third commit is there to show the difference between the old and new version. This is useful for existing module developers to know what to update. If you are familiar with git, you can simply just cherry-pick this commit to finish the migration.

Small Update to MagiskHide
Some devices ships with broken kernels! These kernels will cause magisk.img fail to mount. Previously this will also cause MagiskHide refuse to work, since the hide list is stored in the image. With a small adjustment, MagiskHide now works fine without the requirement of the image to be mounted, but this also means that if you want to hide certain apps in addition to SafetyNet, you will need to manually add them each time you reboot, since the hide list only exists in memory. In other words, if your kernel cannot mount magisk.img, you are forced to run in core-only mode.

Pixel, Pixel, Pixel
If this update is working fine just as expected, I'll finally put full force into working on Pixel (XL) support. I've already done some brief research, hopefully I can pump out official support soon lol.

Below is the changelog from the previous beta:
  • v13.6 -> v14.0
    - [Magic Mount] Fix a typo in cloning attributes
    - [Daemon] Introduce workaround on Samsung device with weird fork behaviors
    - [script] Simplify installation scripts
    - [script] Installation will migrate old or broken stock boot image backups to proper format
    - [script] Fix an issue with selabel setting in util_functions on Lollipop
    - [busybox] Bump version to 1.27.2
    - [MagiskHide] magisk.img mounted is no longer a requirement
    Devices with issues mounting magisk.img can now run in proper core-only mode
The Following 817 Users Say Thank You to topjohnwu For This Useful Post: [ View ]