Attend XDA's Second Annual Developer Conference, XDA:DevCon 2014!
5,779,531 Members 51,137 Now Online
XDA Developers Android and Mobile Development Forum

[MOD][4.1.2 or 4.2.1][GPL] OpenPDroid [v1.0, 2013-01-14]

Tip us?
 
FFU5y
Old
(Last edited by FFU5y; 21st January 2013 at 12:59 PM.) Reason: Updated cm-10.1 patches to handle new change to SmsDispatcher.java
#1  
Senior Member - OP
Thanks Meter 421
Posts: 161
Join Date: Feb 2011
Default [MOD][4.1.2 or 4.2.1][GPL] OpenPDroid [v1.0, 2013-01-14]

What is OpenPDroid?
OpenPDroid is a set of modifications to the Android framework and libraries which allows fine-tuning of the data which applications are able to retrieve about your device, your account, your messages, and more. Specifically, it is a Privacy service provider (using the PDroid 1.51 interface) forked from CollegeDev's PDroid 2.0, which is itself an extension of Syvat's PDroid.

A word to the wise
I'm just adding this for those who are just looking at OpenPDroid, or are trying to work out how to get involved.
Mateor, wbedard and I are currently in the process of moving repositories etc to a github 'organisation' in order to make it much easier for users to know which are the latest patches, where to report issues, etc. I expect this will be done soon (in the next day or so), and I'll post more information then.
Thanks for your suggestions on ways of managing repositories, patches, etc.


What does it do?
When configured using either FFU5y's PDroid Manager or CollegeDev's PDroid 2.0 App (up to v1.52), OpenPDroid intervenes in API interactions by apps (e.g. when an app attempts to retrieve your location, phone number, or contacts) and provides either real data, empty data, fake data, or random data, depending on the user setting. Unlike apps which actually remove permissions, OpenPDroid does not actually modify the permissions that an app has and so is much less likely to cause the app to crash as a result of data access being denied.
Note that CollegeDev's PDroid 2.0 App does not officially support OpenPDroid, so please don't contact CollegeDev for support if you're having problems with PDroid 2.0 App with OpenPDroid. You can post here, and we may be able to help you.
PDroid Manager does officially support OpenPDroid.
The complete list of data and functions controlled by OpenPDroid can be found at the end of this post, in What (specifically) can I control with OpenPDroid?, but at present it is identical to that offered by PDroid 2.0
For a list of issues which have been identified, please see below the Known Issues.

What makes OpenPDroid different to CollegeDev's PDroid 2.0 or Syvat's PDroid
There are a few differentiators, some technical, and some are philosophical or pragmatic. Both are important
Technical
  • Two major security issues are fixed, which allow apps to bypass many or all PDroid or PDroid 2.0 settings. (We are in the process of notifying CollegeDev of these issues, and sample code will not be released for at least a week to give him a chance to fix the problems in PDroid 2.0. Fixes have been added to PDroid in auto-patcher, so if you are using PDroid please consider re-patching your ROM).
  • It is available for Android 4.2.1
  • Database access threading has been rewritten, as the implementation actually caused bottlenecks and didn't protect from simultaneous read-writes (which is generally the goal of this type of code)
  • A bug which was causing 'null pointer exceptions' to occur (but not a crash) is fixed.
  • Other code tidy-up or restructuring which doesn't change functionality
Non-technical
  • The intention of OpenPDroid is to have a PDroid version which is developed in the open, and which welcomes (although doesn't necessarily always adopt) user input - especially code. This means that the source for OpenPDroid versions in progress is on a public repository (Github at the moment), that changes are put into the Github as they are made, and that other people can contribute code to it. We also hope that by having engaging people more in the development process, there will be more eyes trying to understand the code, and so security-related issues will be identified and address more efficiently.
  • We also want to make the discussion/decision making processes for what is in and what is out transparent, so that users can understand why some things are included and others are not, and actually influence the decision-making process.

How do I install it?
There are two 'supported' ways to install OpenPDroid:
  • mateor's auto-patcher (available for OS X, Linux and Windows) is the recommended way if you are not familiar with building from source, and have a supported ROM (CM10, CM10.1, AOKP Jellybean 4.1.2, AOKP Jellybean 4.2.1, AOSP 4.1.2, and AOSP 4.2.1).
  • Patch and build a ROM from source (see How do I compile a ROM with OpenPDroid). Patches are provided for: CM10, CM10.1, AOKP jb, AOKP jb-mr1, AOSP 4.1.2, AOSP 4.2.1
If you are using a stock ROM, then someone may have implemented OpenPDroid into that ROM - and if they have, they deserve huge credit because it is a difficult and unpleasant job.
Currently we are not aware of any ROMs which include OpenPDroid: if you are aware of one, please notify us and we'll list it here.
There are several reasons your preferred ROM may not currently be supported. First, adding support for a ROM in auto-patcher requires building the ROM with the OpenPDroid patches. We are only able to do this if the source is available for the ROM. (Note: strictly speaking, it is possible to build patches for ROMs where source is not available, as for stock ROMs. However, for non-stock ROMs the rate of change makes this fairly unmaintainable).
Second, there are a great many ROMs out there and it is not possible for us to build for all these ROMs. However, if you are able to patch source and build, then you may be able to build a version of your preferred ROM with OpenPDroid support - and even better, contribute patches for your preferred ROM to auto-patcher. For more details about contributing patches to auto-patcher, please contact Mateor.

How do I get the source?
The source is available as patches, or in complete form, from the OpenPDroid repositories[list][*] The source-code patches from the OpenPDroidPatches repository: [url=https://github.com/OpenPDroid/OpenPDroidPatches[/url]. Note that there are two branches: 4.2.1 and 4.1.2. These source patches presently work against numerous roms (including CM 10/10.1, AOKP 4.1.2/4.2.1, AOSP 4.1.2/4.2.1, and potentially others)[*] Complete repositories for the modified sections of the system are likewise located in the OpenPDroid github. There are several repositories and branches; see the section 'What are all these repos and branches?' for more details about the content of the github.

What is the licence?
PDroid was under under the GPL v3 licence (as shown on the Google Code page), and as a result all derivative works (e.g. OpenPDroid) are likewise under the GPL v3 (or later) licence. Thus, OpenPDroid is under the GPL v3 licence, with no additional special conditions. The only way a PDroid derivative can be under any other licence is if the author contacts Syvat, gets him to agree provide PDroid under another licence, and then the author extends it.

Who's work is it?
PDroid was original developed by Syvat (with contributions and ports from others such as Pastime1971 and Mateor), and then CollegeDev extended it to created PDroid 2.0 (previously called PDroid Addon, then PDroid Extension). PDroid 2.0 has been forked by FFU5y, Mateor and wbedard to create OpenPDroid. Thus, OpenPDroid is the product of the work of many.
The following list includes all the contributors I know of for any of these version of the core (in alphabetical order):
  • CollegeDev (PDroid 2.0)
  • FFU5y (OpenPDroid)
  • Mateor (Porting PDroid to 4.x, porting PDroid 2.0 to 4.1.2, OpenPDroid)
  • Pastime1971 (Porting PDroid to 4.x, porting PDroid 2.0 to 4.1.2)
  • Syvat (PDroid)
  • Wbedard (OpenPDroid)

How do I report issues?
The best way to report an issue is to lodge it in github, although you may want to also post in this thread to let others know that the issue exists (and has been lodged). Please ensure you provide details of the issue you are experiencing (i.e. under what circumstances does it happen, what do you observe, etc) and please include a logcat: see how to get a logcat.
If you don't have a Github account, you can post the issue here (and note that you haven't lodged it in Github) so one of the team can add the issue to Github.

I want to contribute
Great! The whole point of OpenPDroid is to encourage community contribution in the form of ideas, testing, code, etc.
Presently, there isn't really a lot (read: any) documentation about how OpenPDroid works. However, this will be forthcoming, as will an API specification for the Privacy service, which will allow you to write your own management application (like PDroid Manager or the PDroid 2.0 App) if you so desire.
Right now, these are the particular areas of the service which would benefit from developer/code contributions.
  • Transient caching of privacy settings read from the database, ideally with destruction of cache entries when they are too old or memory conditions are low
  • Refining the locking code to ensure that reads are not unduly held up by write locks (note: this isn't a problem *yet*, but will be with the inclusion of batch support. More details soon).
  • Other performance optimisations
  • Adding good quality code comments - mainly function descriptions, etc
  • Back-porting of OpenPDroid to pre-4.1.2, so users currently only able to use PDroid can move to OpenPDroid and get the new features.
There are many other things you can work on, including adding whole new options (i.e. security features). If you are planning on creating new security features, however, please lodge the enhancement in github first so it can be discussed. Not every feature which seems like a good idea will be a good fit for OpenPDroid, and it may save some time if a prospective feature can be refined before it is fully coded.

How do I compile a ROM with OpenPDroid
To build a ROM with OpenPDroid, you will need to patch the source code using the provided patches, and then build as you normally would.
To patch the source:
  • Download the most relevant of the source code patch packages attached to this post
  • Extract the patch files from the package
  • On the command line, go to the root of your Android source code: it will contain folders such as 'build', 'frameworks', 'vendor', etc.
  • Patch the source code with each patch file using the following command
    Code:
    patch -p1 < <patch_filename>
You can then build your ROM as you would normally.
If you have previously built the ROM in the folder you are using, or you have previously built PDroid 2.0, Syvat's original PDroid, etc it can be helpful to delete the following directories:
  • out/target/common/obj/JAVA_LIBRARIES/framework_intermediates
  • out/target/common/obj/JAVA_LIBRARIES/framework2_intermediates
  • out/target/common/obj/JAVA_LIBRARIES/telephony-common_intermediates
Not all of these directories will exist for all ROMs, so if you try to delete one and it is missing that is not cause for concern.

What (specifically) can I control with OpenPDroid?
Currently, access to the following data and functions can be controlled (note this list exactly matches PDroid 2.0):
Code:
Data access:
Device ID (IMEI/MEID/ESN)
Phone/voicemail number
Sim Card Serial (ICCID)
Subscriber ID (IMSI)
Source number of incoming calls
Destination number of outgoing calls
Network location
GPS location
Authentication credentials
List of accounts, including identifiers (e.g. gmail e-mail address)
Contact list and details
Call log
Calendar
Read and send SMS
Read and send MMS
Record audio (not phone calls)
Camera
Browser bookmarks and history
System logs
Wifi Info, such as current access point, IP address
Network information (detailed network information such as connection state, IP address, etc)
Android ID (a unique installation ID for your Android installation)
SIM information (your phone operator and country)
Restrict access to iptables command
ICC Access (reading or writing SMS on the SIM or R-UIM)

Actions:
Make phone calls
Start on Boot (or more accurately, get notified when the boot is complete)
Switch Wifi State (turn wifi on and off, change access points)
Switch network state (turn off or on mobile data)

Other
Force online state (always tell the app that the device is online: only needed if denying Wifi info and/or Network info)
Known Issues
Thanks to those users who have posted bug details, we now have a list of known issues. I have tried to indicate which will be fixed in the next release, but it is tentative. These issues can also be found (and discussed) on the github issues list
What are all these repos and branches?
In the github account containing the OpenPDroid repositories, there are numerous repositories. Many of these are OpenPdroid related: but not quite all of them. In addition, not all of them are relevant to each ROM.
The main repositories to be aware of are:
platform_frameworks_base
This contains the framework/base/* code, which constitutes the most important guts of OpenPDroid in 4.1.2 and 4.2.1. Other projects sometimes use the name android_frameworks_base for this.
platform_frameworks_opt_telephony
This contains the framework/opt/telephony code, which was split out from framework/base/* in 4.2.1.
platform_build
This contains the build code, mostly makefiles, and is present in 4.1.2 and 4.2.1.
platform_libcore
This contains core operational code outside of the framework (e.g. executing shell tasks, etc).
platform_packages_apps_mms
This contains the code for the Mms app, for 4.1.2 and 4.2.1.
platform_packages_apps_mms
This contains the code for the Mms app, which is modified to affect the ability of apps to send and receive MMS.
platform_packages_apps_videoeditor
This contais the code for the VideoEditor app. The only change here is including 'framework2', which is only required for AOSP 4.1.2.
platform_system_core
This contais a range of important core code, and is modified only for AOSP 4.1.2 to include 'framework2.jar' in the paths included at boot time.

The branches used include two key naming components. First, the rom:
e.g.
cm-10.1* = CyanogenMod 10.1
cm-jellybean* = CyanogenMod 10, cutting edge
cm-jellybean-stable* = CyanogenMod 10, stable release
aokp-jb-mr1* = AOKP 4.2.1-based
aokp-jb* (without the mr1) = AOKP 4.1.2-based
jb-mr0* = AOSP 4.1.2
jb-mr1* = AOSP 4.2.1
pa-jellybean* = ParanoidAndroid (currently buggy)

then the OpenPDroid version:
-openpdroid = the current stable release of OpenPDroid
-openpdroid-devel = the current development line of OpenPDroid

For development, the jb-mr1-release-openpdroid-devel is where I do my initial development, and I think expand it to be on the other -devel branches once I've tested it a bit.

FAQ
How do I include PDroid Manager in the ROM I am building so it can update from Google Play: see here
How do I include PDroid Manager in the ROM I am building so it is compiled during build: see here: TamCore provided a fix for the problem he mentioned, and it has been merged.
How do I build using the bleeding-edge (devel) code: see here, although there is a bit more to it than that. I'll provide extra details when time permits, but the important stuff is in this thread.


I have a question or issue not covered in this post
We'll be adding more to this post in the next day or two. Please be patient =)

Note to mods
I wasn't sure if this belonged in this forum, or in 'Apps and Games' but since it is not trivial to install I figured this was a safer bet.
The Following 115 Users Say Thank You to FFU5y For This Useful Post: [ Click to Expand ]
 
FFU5y
Old
#2  
Senior Member - OP
Thanks Meter 421
Posts: 161
Join Date: Feb 2011
reserved
The Following 10 Users Say Thank You to FFU5y For This Useful Post: [ Click to Expand ]
 
JunDavis
Old
#3  
JunDavis's Avatar
Senior Member
Thanks Meter 135
Posts: 553
Join Date: Jun 2012
Location: Crestview
Thank you!!!! I'll test

Sent from my LG-P920 using xda app-developers app

Code:
System.out.println("Java is your friend!");
 
mateorod
Old
(Last edited by mateorod; 30th March 2013 at 11:45 PM.)
#4  
Recognized Developer
Thanks Meter 3508
Posts: 1,976
Join Date: Nov 2011
Location: New Orleans

 
DONATE TO ME
Announcement: OpenPDroid build patches, as well as the entire history of the PDroid framework, have been rebuilt and restructured and pushed to our github page. There are build patches for all Android versions from 2.3 to current. You can also take a look at the commit history of the original PDroid framework by svyat, from initial commit to the point where he halted development, 1.32 for gingerbread.

OpenPdroid for Android 4.2.1/4.2.2

These are the current build patches for Android 4.2.1 and 4.2.2 and should generally reflect the current state of the jb-mr1-release-openpdroid branch. The latest commist are not always reflected in the build patches, bleeding-edge development is in the -devel branches.

We realize that the patch/branch situation is not perfect, it is work in progress.

These build patches should work for most roms. As of their creation (1/17/13) they applied cleanly to AOKP, AOSP, Cyanogenmod, ParanoidAndroid and others.

The current best advice as to how to apply/remove them is below. Some easier application methods are being discussed.
To Apply

Code:
cd ~/android/system/build; git checkout -b pdroid; patch -p1 < ~/openpdroid_4.2.1_build.patch
cd ~/android/system/libcore; git checkout -b pdroid; patch -p1 < openpdroid_4.2.1_libcore.patch
cd ~/android/system/packages/apps/Mms; git checkout -b pdroid; patch -p1 < openpdroid_4.2.1_Mms.patch
cd ~/android/system/frameworks/base; git checkout -b pdroid; patch -p1 < openpdroid_4.2.1_frameworks_base.patch
cd ~/android/system/frameworks/opt/telephony; git checkout -b pdroid; patch -p1 < ~/openpdroid_4.2.1_frameworks_opt_telephony.patch
cd ~/android/system; . build/envsetup.sh; brunch <DEVICE_TARGET>
To Remove
Code:
cd ~/android/system
rm -rf out/target/common/obj/JAVA_LIBRARIES/framework_intermediates \
out/host/common/obj/JAVA_LIBRARIES/layoutlib_intermediates \
out/target/common/obj/JAVA_LIBRARIES/telephony-common_intermediates \
out/target/common/obj/APPS/Mms_intermediates
[OR]
Code:
cd ~/android; make clobber
and to reset your source tree**

Code:
cd ~/android/system/build; git checkout . ; git clean -df
cd ~/android/system/libcore; git checkout . ; git clean -df
cd ~/android/system/frameworks/base; git checkout . ; git clean -df
cd ~/android/system/frameworks/opt/telephony; git checkout . ; git clean -df
cd ~/android/system/packages/apps/Mms; git checkout . ; git clean -df
cd ~/android/system; repo abandon pdroid
If OpenPDroid intrigues you, yet the thought of applying all these patches gives you pause, there is a tool that can attempt to patch the framework of an already compiled rom called The Auto-Patcher.

OpenPDroid is an open source fork of the PDroid framework, as written by Svyat and expanded by CollegeDev. We are always looking for contributors, come see us on XDA or submit a pull request.
Support Beloko Games' source ports of Doom and Quake 1-3, Jedi Knight, etc.

He is hurting from piracy...I need Doom 3 on my phone, you guys. Buy them, ok?
The Following 12 Users Say Thank You to mateorod For This Useful Post: [ Click to Expand ]
 
decimalman
Old
#5  
decimalman's Avatar
Senior Member
Thanks Meter 2837
Posts: 1,041
Join Date: Nov 2012

 
DONATE TO ME
I'm glad to see there's a more open PDroid project happening. I'm not sure what I'll be able to contribute, but I'll find something to help out with.

Looks like I'm going to have to spend some quality time with the CM10.1 codebase tonight.
 
FFU5y
Old
#6  
Senior Member - OP
Thanks Meter 421
Posts: 161
Join Date: Feb 2011
Decimalman:
If you look in my github (http://github.com/wsot) you will find there are forks of all the modified parts of Android, and these include forks from cm-10.1 (branches called cm-10.1-openpdroid). You can pull directly from these repositories into your existing repositories if you would prefer to do that, rather than apply patches to the source.

For cm-10.1, the repositories you'll care about are:
* platform_frameworks_base
* platform_libcore
* platform_build
* platform_packages_apps_mms
* platform_opt_telephony_common

Each of these has a branch for cm-10.1-openpdroid with the changes included.
The Following 3 Users Say Thank You to FFU5y For This Useful Post: [ Click to Expand ]
 
M66B
Old
#7  
M66B's Avatar
Recognized Developer
Thanks Meter 9375
Posts: 5,257
Join Date: Aug 2010
Location: Dordrecht

 
DONATE TO ME
This is great work guys, thanks!

I will test it and if everything works, I will release the next extended CM10/FXP ROM with it.
Is there already a solution for including PDroid Manager and still being able to get updates from the market?

I have one question, did you fix this bug too?

Is there any plan to cooperate with CollegeDev?
It would be a pity if development for PDroid would be fragmented.
Looking for a reliable software engineer ?
 
FFU5y
Old
#8  
Senior Member - OP
Thanks Meter 421
Posts: 161
Join Date: Feb 2011
Quote:
Originally Posted by M66B View Post
I will test it and if everything works, I will release the next extended CM10/FXP ROM with it.
Excellent!

Quote:
Originally Posted by M66B View Post
Is there already a solution for including PDroid Manager and still being able to get updates from the market?
If you include PDroid Manager as-is from the PDroid Manager thread, then it has the same signature as the one in the market. Because of that, it should be able to update from the market.

Quote:
Originally Posted by M66B View Post
I have one question, did you fix this bug too?
No, thanks for pointing that out. I've updated the source and pushed it to Github with that change. It'll be in the next release, which I don't imagine will be all that far away.

Quote:
Originally Posted by M66B View Post
Is there any plan to cooperate with CollegeDev?
It would be a pity if development for PDroid would be fragmented.
We have not discussed with CollegeDev any kind of collaboration at this stage. Because both OpenPDroid and PDroid 2.0 core are GPL licensed, I imagine there will be code migration in both directions as things are fixed and improved.
The main reason that OpenPDroid has appeared (rather than us just making a 4.2.1 port of PDroid 2.0) is because long periods of development of PDroid 2.0 went on without the code being available, and so there was really no scope to collaborate.

The reason we are planning to produce an OpenPDroid API document is essentially so that there is a clear (and shared) idea of what can be expected from a minimal PDroid implementation, which both PDroid 2.0 and OpenPDroid cover at this stage. That way even if additional features are added (or someone creates a whole new PDroid implementation from scratch) there can be a shared interface to allow interoperability of management apps (like PDroid Manager).

I think there will be some 'fragmentation', but our hope is that there can be a common API between the versions that users will not have to suffer as a result, and can continue to use their preferred management app, and so forth.
Whether than will happen, I can't say yet.
The Following 4 Users Say Thank You to FFU5y For This Useful Post: [ Click to Expand ]
 
TamCore
Old
#9  
TamCore's Avatar
Senior Member
Thanks Meter 1057
Posts: 655
Join Date: Jul 2010
Location: Düsseldorf

 
DONATE TO ME
Thank you! That's what I've been waiting for. I'll integrate this in OpenSensation2 asap
Blog // Twitter // Google+ // github // something else
 
M66B
Old
#10  
M66B's Avatar
Recognized Developer
Thanks Meter 9375
Posts: 5,257
Join Date: Aug 2010
Location: Dordrecht

 
DONATE TO ME
Quote:
Originally Posted by FFU5y View Post
If you include PDroid Manager as-is from the PDroid Manager thread, then it has the same signature as the one in the market. Because of that, it should be able to update from the market.
Good! Could you make a 'latest' URL available somewhere (github?), so I can integrate it in my build process?
Looking for a reliable software engineer ?

Tags
openpdroid, pdroid, pdroid 2.0
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes