FORUMS

[DEV] [R&D] Open Android Patcher (OAP) & MIUI Ex

5,836 posts
Thanks Meter: 7,716
 
By CosmicDan, Senior Member on 16th September 2018, 08:29 AM
Post Reply Email Thread
Announcement from CosmicDan: OP overhauled (finally), check it out!




News

CURRENT PROJECT STAGE: Late-Initial R&D
PENDING PROJECT STAGE: Prototyping [No ETA]

2019.02.16
  • Overhauled this thread to actually represent the current project (all the old stuff is in a spoiler at the bottom)
  • I made a cool logo! Well, let's consider it a first-draft... if anybody thinks they can do better then you're welcome to share! But please don't be offended if I say "I don't like it" (I can be picky )
  • A GitHub organization for the project has been created! There's only one public repo right now, which is the Linux/WSL-only 'preptool' and has no documentation.
  • As mentioned above, there is an OAP PrepTool repo released. Alpha though - right now it can unpack and de-opt (deodex) the latest Pie-based MIUI ROM's for Poco, and probably some other devices (it makes a few assumptions though). It's tested in WSL (Ubuntu on Windows) only, but there's no reason why it wouldn't work in any "real" Linux/GNU distro. Note that the repack and reodex of ROM's is not yet finished. But I welcome anybody with a different device to test the unpack/deodex process and report any errors you get from it.
  • I've been on a break for the past few weeks as I am quitting smoking - successfully (yay me!) - but want to keep stress to a minimum to maximize my chances of success
  • Haven't heard from Lanchon (creator of DexPatcher, an essential project for OAP) for a couple weeks now, but he's a very busy guy who was always super helpful and cool in the past so I don't want to pester him again just yet. He is aware of what we're doing here though and said he'll get back to me when he can



About

Whew, where to start? The project is still in it's infancy so this is a lengthy 'About' section as I try to convey the vision of this project to any person who might be interested in helping out. Yeah it's a lot to read - but surely worth reading once. In future I'll say in the News section above if any specific section of the thread was updated.

While I consider myself an Android power-user and veteran developer ( [brag] my past XDA and freelance work has landed me several contract in Android development [/brag] ) I'm not ashamed to say I am a huge MIUI fan and prefer stock firmware over any custom ROM's (stability is essential to me). It's a pretty great stock ROM that I missed for a long time - generally reliable yet feature-rich. However, it does lack some functions in key areas (which custom ROM's have enjoyed for a long time) and also has a lot of small issues that Xiaomi haven't acknowledged or fixed for whatever reason (probably a cost-benefit analysis says it's not worth fixing - you know, that typical big business stuff that we're simply not privy to because competitors could use it to get an advantage or something).

I intend to improve on MIUI out of my own love for the system, the memory and nostalgia of my early days in Android/MIUI modding ( [brag] such as this or this, the two projects that launched my professional Android journey [/brag] ) has me inspired to Make MIUI Great Again. However, before working on a another modded MIUI ROM, I wanted to do things right by the community and make things as future-proof as reasonably possible: Make a build system (or "kitchen"), make it developer-friendly and modular, and make it all free and open-source.

This project/initiative has been dubbed the Open Android Patcher (or OAP for short) and, while specific to MIUI for the moment as that's the template used during design, OAP has long-term goals to be generic and modular for any stock-ROM modding.

To simplify, the idea is pretty straight-forward - make a tool kit that lets developers write patches for stock ROM's via mods that are similar in principle (and how they're written) to Xposed mods. Unlike Xposed mods though, the mods are baked-in to the ROM itself rather than performed at runtime. This means that, unlike Xposed, Safetynet still works, performance stays 100%, and NO SECURITY RISKS. However, obviously it won't be as flexible as Xposed - if you want to add or remove a mod you will need to rebuild and/or reflash the firmware and/or changed files.

The end-goal is to do a similar thing as what Xiaomi.eu, Mi-Globe, etc. all do but *NOT* keep the build system a well-guarded secret (I've reached out to *all* these guys and they either ignored me or politely refused to be a part of anything open). That is, everything related to OAP will remain 100% open and transparent (unless a developer makes a mod and ignores the GPL license), welcoming to contributions from others via usual GitHub PR's. There will, however, be many more features than other MIUI-modded ROM's - features which you usually only see in AOSP forks (Omni, RR, etc.) or Xposed modules; part of my previous professional Java work was extensive reverse-engineering of Android applications and ROM's (rebranding Chinese devices for global markets, most Chinese ODM's prefer we reverse-engineer it rather than giving us their source code lol) and there are a lot of possibilities that I want to explore here which I am very confident in doing so. Most of all, though, I want to make it easier for other developers to contribute mods too. After all, that's what open development is all about, right? The thanks and feel-goods we get from others!

I will be sharing regular updates as things develop, and continually seeking feedback from prospective developers and users.

All sources relating to the project will be in repo's at the GitHub.com/OpenAndroidPatcher organization.



Project Goals/Status

A very-draft kind-of-Roadmap. This will be improved soon (it's one of the tasks below, heh).

Key/Legend:
[#] Ongoing/current task in initial/early R&D stage
[!] Ongoing/current task in initial/early R&D stage but might be abandoned for whatever reason (e.g. too difficult and not worth the time unless I attract some genius talent to help, or it's too early to tell if it's even possible or useful)
[@] Ongoing/current task in deep development and/or with unknown or indefinite completion time
[+] Ongoing/current task nearing completion
[-] Future task, blocked by a previous incomplete task
[?] Future "maybe" task
  • [@] Experiment with MIUI fixes and improvements
  • [@] Research bugs and issues that Xiaomi/Poco haven't addressed
  • [@] Create the Java-based OpenAndroidPatcher (OAP) build system (kitchen)
  • [#] Integrate and adapt DexPatcher for OAP (so we can write stock ROM mods in pure Java with all benefits of IntelliJ IDEA)
  • [-] Create a more low-level patching system for advanced patches (will probably a wrapper around dexlib2)
  • [-] Create the OAP framework/library. This is a framework jar + IDE library jar that OAP patches link against at runtime and compile time respectively which aims to take a lot of the boilerplate and tediousness out of writing ROM mods (e.g. Designing an settings UI's, working with apk/framework resources, lots of other shortcuts and helpers that are helpful or necessary when patching stock ROM's)
  • [-] Design the "OAP Tweaker" app, to act as a centralized app that handles all mod configuration UI's automagically
  • [-] Formalize "MIUI Ex" - the flagship 'product' (for want of a better word) of OAP
  • [#] Make a nifty (but actually useful) website
  • [#] Make a public Freedcamp project for easier collaboration and more transparency
  • [?] Make a Discord server or Telegram group for senior developers to chat directly on
  • [?] Easy debugging and testing of OAP patches on a live development device
  • [!] Create a hosted build server
  • [!] Create a TRUE OTA system for automatic delta (incremental) updates; none of that "Notification that links to a webpage with full ROM ZIP to manually download and flash" that so many custom ROM's do
  • [?] Completely replace the build system with a Magisk-powered dynamic patching system, making a custom OTA/build server obsolete. This is a HUGE task but an ultimate-awesome one that I'd love to see one day.



What's this thread for?

TL;DR: A place for other interested developers to show their interest and share ideas. When the project is a bit more mature, a better platform of communication will be established.

Honestly, this thread was written a bit prematurely - but obviously I don't want it locked. In retrospect I would not have started it so early - I did not expect it to blow up with popularity, since then the original goal has changed significantly from "a custom MIUI build" to "a universal and modular stock-ROM patching kitchen initially aimed at MIUI". The purpose of this thread is now fairly vague because development is still in too early stages. I can't really accept any serious help from others yet because the project is still too immature to allow any compartmentalization or delegation of the tasks. In other words - for those trained in Project Management vocations - most of the important tasks right now are all along the critical path. Those that aren't on the critical path are really complicated and I've not had any knowledgable volunteers yet.

So anyway, this thread is mostly used as a blog for me and a place for everyone to spin ideas on how to make MIUI awesome. Anybody with an interest in the project is free to ask and talk about the development or ideas, as long as it's at least loosely related or might be of help to the project. I'm pretty active on XDA, so as long as you don't hijack the thread for some unrelated help on other Poco stuff, I'm happy for anybody to post and happy to answer - even non-developers and dev novices haha (as long as you're not rude, spammy of off-topic of course - at best you'll be silently ignored).

With that said, thank you for stopping by... and check back soon!


Old thread content below (along with the highly-experimental-and-laggy-as-heck port of Mi 8 MIUI to Poco).

------- [START OLD THREAD SPOILER]
Flavors

Xiaomi.eu for Beryllium (Poco) - port from Dipper (Mi 8)

NOT FOR DAILY USE!
  • Clean port from Xiaomi.eu, all essentials appear working (Camera, Calls, Wifi, Sound, etc.)
  • [NEW] FBE (File-based encryption) enabled. Requires custom TWRP build or FBE-disable ZIP to remove (see Download section)
  • [BUG] FM Radio not working (blocked by kitchen development, see GitHub issue #1)
  • [BUG] MicroSD does not appear on PC when in USB File Transfer mode. External storage works fine otherwise, though.
  • [ISSUE] Performance is poor (UI jank).
  • [ISSUE] GPS/Location MIGHT be worse than stock official. Same reason as above (Issue #1).
  • [ISSUE] Notch cut-out and status bar size/alignment is a little bit too big.

That's the only flavor for now



Known Issues (all flavors)
  • Device is janky, sometimes even pausing for a couple seconds.
    The only current flavor is an experimental port which has issues. This was only built as an experiment for people to try-out Mi8 features/blobs (e.g. Camera).



Downloads

Main ROM's

Latest builds at GitHub - https://github.com/cosmicdan/MIUI_Kitchen/releases


Extra's

The following extras can be found on DevBB and also GitHub (but I will only add them to a GitHub release if it has been updated - so it will be easier to find these downloads on DevBB instead):
  • A custom TWRP build with MicroSD fixed, FBE (file-based encryption) support, and extra mounts for Backup/Restore
  • A flashable ZIP to enable FBE on the current ROM. Note that FBE is already enabled by default on some flavors (see above). Requires Magisk 17.1+ (flash Magisk before or after - order doesn't matter - as long as both come after the ROM).
  • A flashable ZIP to disable FBE on the current ROM if you don't want any encryption. Requires Magisk 17.1+ (flash Magisk before or after - order doesn't matter - as long as both come after the ROM).



Install Instructions

Prerequisites for ALL first-time users:
  1. [REQUIRED] Unlock your bootloader. Do not ask about unlocking bootloader in this thread, it is off-topic.
  2. [DOWNLOAD] Download the desired ZIP from GitHub (there is only one flavor now, so just get whatever is latest)
  3. [RECOMMENDED] Flash my TWRP which has encryption changed to FBE, working MicroSD, and extra partitions added for backup. Nothing special here, just the usual 'fastboot flash recovery twrp-*' command. Do not ask for help about how to fastboot flash here.
  4. [RECOMMENDED] Make a FULL backup of your current ROM in TWRP. These firmware's contain kernel, system, and vendor - just like official MIUI ROM's - so a TWRP backup should allow you to quickly revert if you need to.

Moving from ANY other ROM?
  1. [REQUIRED] Factory Reset (swipe) or "Format Data" (button). If encrypted with stock-style encryption, the full Format Data is essential to clear the incompatible old-fashioned full-device encryption.
  2. [FLASH] Copy the ROM ZIP to device and flash in TWRP
  3. [REQUIRED] Flash Magisk 17.1 (or later).
  4. [OPTIONAL] Flash the FBE-Disabler ZIP if you want to completely disable encryption. This will modify the Vendor partition, so will apply to any GSI's you flash too.
  5. [RECOMMENDED] Boot the ROM and wait for it to boot (will take longer than you are used to). Reboot it again once booted, and keep rebooting until boot becomes fast (fast as stock) again. See Known Issues above for why this is important.

Upgrading, downgrading or reinstalling a CosmicDan MIUI Build ROM?
  1. [OPTIONAL] Factory Reset in TWRP
  2. [FLASH] Copy the ROM ZIP to device and flash in TWRP
  3. [REQUIRED] Flash Magisk 17.1 (or later).
  4. [OPTIONAL] Flash the FBE-Disabler ZIP if you want to completely disable encryption. This will modify the Vendor partition, so will apply to any GSI's you flash too.
  5. [OPTIONAL] Wipe Dalvik/Cache if you encounter a boot loop. This is a very rare requirement - Android is smart enough to know when Cache is old and can recycle it. If you do wipe cache, be aware of the "Reboot again" optimization stuff again, as from a clean flash.



Credits and Thanks

- Xiaomi.eu for the fantastic conversions of MIUI China ROM's
- All those mentioned in README.md in the Kitchen source for making the tools it depends on (I will list them here once the kitchen development settles down)


------- [END OLD THREAD SPOILER]


XDA:DevDB Information
Open Android Patcher (OAP) & MIUI Ex, ROM for the Xiaomi Poco F1

Contributors
CosmicDan
Source Code: https://github.com/OpenAndroidPatcher

ROM OS Version: 9.x Pie
ROM Firmware Required: Unlocked BL
Based On: MIUI

Version Information
Status: Alpha

Created 2018-09-16
Last Updated 2019-02-15
The Following 97 Users Say Thank You to CosmicDan For This Useful Post: [ View ] Gift CosmicDan Ad-Free
16th September 2018, 08:29 AM |#2  
CosmicDan's Avatar
OP Senior Member
Flag Sydney
Thanks Meter: 7,716
 
Donate to Me
More
Reserved
16th September 2018, 08:34 AM |#3  
SmallTarzan's Avatar
Senior Member
Flag Bratislava
Thanks Meter: 129
 
Donate to Me
More
Still first.

Gonna design it now.
The Following 3 Users Say Thank You to SmallTarzan For This Useful Post: [ View ] Gift SmallTarzan Ad-Free
16th September 2018, 08:42 AM |#4  
CosmicDan's Avatar
OP Senior Member
Flag Sydney
Thanks Meter: 7,716
 
Donate to Me
More
Quote:
Originally Posted by SmallTarzan

First.

And I still hate you.

xD

Just discovered another bug, MicroSD does not appear in File Transfer mode - only storage. Which is really weird because it appears fine in literally everything else.

Updated OP.
The Following 5 Users Say Thank You to CosmicDan For This Useful Post: [ View ] Gift CosmicDan Ad-Free
16th September 2018, 09:01 AM |#5  
madsponge26's Avatar
Senior Member
Flag Pilipinas
Thanks Meter: 3,071
 
Donate to Me
More
Nice to see this..
16th September 2018, 09:01 AM |#6  
Senior Member
Thanks Meter: 17
 
More
Trying, congratulations on the "Kickstart" thread
16th September 2018, 09:16 AM |#7  
Vatumb's Avatar
Senior Member
Thanks Meter: 97
 
More
Booted
16th September 2018, 10:07 AM |#8  
narora9999's Avatar
Member
Flag Ludhiana
Thanks Meter: 50
 
More
Is it based on Oreo 8.0 or 8.1? I need an 8.0 miui rom to install xposed so asked.
16th September 2018, 10:25 AM |#9  
Member
Flag Razhok'
Thanks Meter: 59
 
Donate to Me
More
*****RESERVED*****
Edit 1
First of all I'd like to thank cosmic dan for his support and love for our still infant growing poco
The ROM boots up fine just takes little more time than usual because (read mentioned by Dev in post) you'll probably have to reboot 15 times and devices gets snappier and faster after every boot
For those who wants to know what's a xiomi.eu flavour it's simply MIUI without all the bloatware and most hated Chinese apps
I encourage you use the device for sometime after few reboots to help settle it down but for daily driver I'll still wait for the next build. XD
The Following User Says Thank You to $hivaM Manav For This Useful Post: [ View ] Gift $hivaM Manav Ad-Free
16th September 2018, 10:28 AM |#10  
thebenefactor's Avatar
Senior Member
Thanks Meter: 87
 
More
Gr8 start.
16th September 2018, 11:29 AM |#11  
Senior Member
Thanks Meter: 701
 
More
Hi,
Nice to see you working on this...
However, you mention that you think MIUI lacks some features. But, unless I have overseen it, you do not tell what features we will get with your ROM... so what we will get in trade for the little things not working
Could you be more specific so I can see if you're adding that feature I am looking for (that I could add with XPosed but Xposed does not work with MIUI based on 8.x... and a magisk module isn't available for that function).

Regards.
Post Reply Subscribe to Thread

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes