Carrier IQ Profile collection for EFF

Search This thread

TrevE

Retired Recognized Developer
Apr 27, 2007
2,031
3,659
androidsecuritytest.com
Hey guys, an EFF volunteer - Jered Wierzbicki - has created an application called IQIQ to decode Carrier IQ profiles back to XML. Their public git repo is in the above linked article. In a nutshell, Profiles contain what data is sent back at what interval and to where.

We are now able to see sets of metrics called from phones. The problem is right now we only have default profiles. The EFF is looking to collect as many profiles from as many different devices as possible trying to find real metrics.


Github Proof of concept code for a "profile scraper": https://github.com/TrevE/IQTool
(right now it scans, but only works for automatic sending of archive.img from tmobile, but it should be for root or non root)
If you can contribute go for it! It would help to have a "one click profile sender" out there for everyone.




Possible Methods

FINDING .PRO FILES
We are not too sure yet where updated profiles get stored, so for the time being root users can run the following to search out all .pro files on your disk.

Code:
adb shell busybox find / -iname "*.pro"

It will likely be some sort of IQProfile.pro, CIQProfile.pro, defaultprofile.pro. Once you locate it just:

Code:
adb pull /full/path/to/profile.pro .

Other strategies, such as grepping for a characteristic string like " CONSTANT " across the whole filesystem, might also turn out to be useful. Use this thread to experiment with and improve such techniques!


Waiters suggests a command reference:
maybe something like this (which could take hours to run...)

Code:
grep -r "CONSTANT PROPID" /





Getting archive.img from non-embedded CIQ
We don't yet know if all profiles will be .pro files, or if they'll sometimes be embedded inside of other things. If you are on tmobile the profile information is potentially contained in an archive.img file. The file could be world readable, so you might not even need to be rooted.


WARNING ABOUT ARCHIVE.IMG FORMATS:
Please be warned that sensitive data could be in this archive.img file such as URLs, IMEI, SMS metadata, etc.. EFF will always do its best to keep archive.img files confidential, but please DO NOT send them if there may be any private information on the handset you are working with

Example of where Tmobiles CIQ archive is:
Code:
adb pull /data/data/com.carrieriq.tmobile/app_iq_archive/archive.img .




KNOWN STOCK MD5SUMS
Were really trying to find some new profiles containing the pushed metrics, so to check if you are looking at a "stock" profile or not (this only applies to if you find a .pro file)

Code:
adb shell busybox md5sum /path/to/profile.pro

If it matches one of these md5sums (number on the left) it is likely a known "stock" profile.
Code:
e37a4a8e3ea6d6aa4b7423a462541fa9  att-galaxy-s2-defaultProfile.pro
2618eaa2e3310ec36e1b86f8b643c5fa  htc-amaze-tmob-defaultProfile.pro
a6886135d2d1ea423d4edde389fe1794  htc-evo-sprint-iqprofile.pro
2618eaa2e3310ec36e1b86f8b643c5fa  tmob-defaultProfile.pro


SUBMITTING PROFILES

If you would like to submit your profile or archive.img to EFF so CarrierIQ metrics on what operator can be better understood, please send that data off to iqiq@eff.org . It would be very helpful to them to include phone model and network it was pulled from as well.
Thanks for all your help guys!
 
Last edited:

AdamOutler

Retired Senior Recognized Developer
Feb 18, 2011
5,224
9,826
Miami, Fl̨̞̲̟̦̀̈̃͛҃҅͟orida
Any other names?
Code:
adam@Adam-Desktop:~/IQIQ$ adb shell
/ # busybox find / -iname "*.pro"
/data/data/pl.aygorund.littlepiano.pro
/ #
I'm sure "little piano" which was free on amazon 2 days ago is not Carrier IQ related.
Note:This is an AT&T Infuse 4G with a 4 month old custom kernel.
 
Last edited:

Orical

Senior Member
May 22, 2011
2,211
731
Boston
I'm just glad it's out in the open, there's enough crap in the world to have to worry about, if your banking from your phone (no chance I would have in the first place but) there's a real threat and a legit reason to go in personally to change all of your accounts and who in the world has had all pass access to look at it for how long.

Thanks for the post man, this is that great eye opener that needed to happen I just hope nothing happens to anyone because of Carrier IQ. It's good to see there are others that are taking this seriously.
 

sgt. slaughter

Retired Forum Moderator
Jun 12, 2010
5,715
3,088
Raleigh
tried this on my sprint evo3d and got a odd response...

when I go into adb shell and type: busybox find / -iname "*.pro"
I get the following:
/system/etc/iqprofile.pro
find: /data/DxDrm/fuse: Permission denied

Thought this was odd since im fully rooted s-off here and would be denied access to something, no? The first few times I did it straight from the comand line and that resulted in only outputting the "...permission denied" line shown above. When I did "adb shell" first by itself and then did the second part of the command it gave me the iqprofile.pro....

any thoughts?
 

PromaneX

Senior Member
Jan 31, 2010
119
23
Manchester
tried this on my sprint evo3d and got a odd response...

when I go into adb shell and type: busybox find / -iname "*.pro"
I get the following:
/system/etc/iqprofile.pro
find: /data/DxDrm/fuse: Permission denied

Thought this was odd since im fully rooted s-off here and would be denied access to something, no? The first few times I did it straight from the comand line and that resulted in only outputting the "...permission denied" line shown above. When I did "adb shell" first by itself and then did the second part of the command it gave me the iqprofile.pro....

any thoughts?

I had to go into a shell, do the su command to gain root, and THEN do the search.
 

TrevE

Retired Recognized Developer
Apr 27, 2007
2,031
3,659
androidsecuritytest.com
Any other names?
Code:
adam@Adam-Desktop:~/IQIQ$ adb shell
/ # busybox find / -iname "*.pro"
/data/data/pl.aygorund.littlepiano.pro
/ #
I'm sure "little piano" which was free on amazon 2 days ago is not Carrier IQ related.
Note:This is an AT&T Infuse 4G with a 4 month old custom kernel.


Not sure, does the Infuse have CIQ?


tried this on my sprint evo3d and got a odd response...

when I go into adb shell and type: busybox find / -iname "*.pro"
I get the following:
/system/etc/iqprofile.pro
find: /data/DxDrm/fuse: Permission denied

Thought this was odd since im fully rooted s-off here and would be denied access to something, no? The first few times I did it straight from the comand line and that resulted in only outputting the "...permission denied" line shown above. When I did "adb shell" first by itself and then did the second part of the command it gave me the iqprofile.pro....

any thoughts?


That system/etc/iqprofile.pro might get updated on the e3d, were not sure yet how that works. Can you run busybox md5sum on that file?

I will post known hashes for stuff EFF has already (default profiles).
 

sgt. slaughter

Retired Forum Moderator
Jun 12, 2010
5,715
3,088
Raleigh
Not sure, does the Infuse have CIQ?





That system/etc/iqprofile.pro might get updated on the e3d, were not sure yet how that works. Can you run busybox md5sum on that file?

I will post known hashes for stuff EFF has already (default profiles).

just to be sure i did this right, heres what i did:

adb shell
/system/xbin/busybox md5sum system/etc/iqprofile.pro

this output the following:
a6886135d2d1ea423d4edde389fe1794 system/etc/iqprofile.pro

and btw im running the stock OTA with full CIQ and stock kernel here.
 
Last edited:
S

shaky153

Guest
Not sure, does the Infuse have CIQ?





That system/etc/iqprofile.pro might get updated on the e3d, were not sure yet how that works. Can you run busybox md5sum on that file?

I will post known hashes for stuff EFF has already (default profiles).

TrevE Im not sure if you mentioned this but I have been working on kernel and I came across code in kernel to do with carrier IQ
 

AdamOutler

Retired Senior Recognized Developer
Feb 18, 2011
5,224
9,826
Miami, Fl̨̞̲̟̦̀̈̃͛҃҅͟orida
Not sure, does the Infuse have CIQ?





That system/etc/iqprofile.pro might get updated on the e3d, were not sure yet how that works. Can you run busybox md5sum on that file?

I will post known hashes for stuff EFF has already (default profiles).



Voodoo says it does. I'm working on the grep search now


Code:
 Voodoo Carrier IQ Detector report:

Build fingerprint:
samsung/SGH-I997/SGH-I997:2.3.6/GINGERBREAD/UCKJ4:user/release-keys



Test for: Linux kernel interfaces
(KERNEL_INTERFACES, confidence 50)

    nothing found


Test for: Android logcat debugging log
(LOGCAT, confidence 100)

    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    D/IQService(  363): Save Success test file:/sdcard/Android/netstate.txt
    found:    W/dalvikvm(15518): No implementation found for native Lcom/carrieriq/iqagent/client/NativeClient;.clientInit (Ljava/lang/Object;)I
    found:    W/dalvikvm(15518): No implementation found for native Lcom/carrieriq/iqagent/client/NativeClient;.clientInit (Ljava/lang/Object;)I


Test for: Linux kernel drivers
(KERNEL_DRIVERS, confidence 50)

    nothing found


Test for: System services
(SERVICES, confidence 70)

    found:    12	com.carrieriq.iqagent.service.IQService: [com.carrieriq.iqagent.interfaces.IService]


Test for: ROM binaries and daemons
(SYSTEM_BINARIES, confidence 70)

    found:    /system/xbin/iqbridged
    found:    /system/bin/iqmsd
    found:    /system/lib/libiq_service.so
    found:    /system/lib/libiq_client.so


Test for: ROM configs
(ETC_CONFIG, confidence 0)

    nothing found


Test for: Packages
(PACKAGES, confidence 70)

    nothing found


Test for: Running processes
(RUNNING_PROCESSES, confidence 200)

    found:    system    363   252   142180 16324 ffffffff 00000000 S android.iqd


Test for: Suspicious classes
(SUSPICIOUS_CLASSES, confidence 0)

    found:    com.carrieriq.iqagent.service.receivers.BootCompletedReceiver


Test for: Linux kernel dmesg log
(DMESG, confidence 100)

    nothing found


-- 
Voodoo CarrierIQ Detector 1.4
 

TrevE

Retired Recognized Developer
Apr 27, 2007
2,031
3,659
androidsecuritytest.com
I made a quick and dirty android app for searching / sending profiles. I will be polishing it up more, anyone else feel free to jump in here to. The more grep strings or other locations these profiles are found will be added in here.


Right now it works only for tmobile flawlessly. But it will work for non rooted tmobile users, so if you feel comfortable send this to everyone on tmo you know to get the ball rolling. There is a compiled copy in bin\

https://github.com/TrevE/IQTool
 
Last edited:

TrevE

Retired Recognized Developer
Apr 27, 2007
2,031
3,659
androidsecuritytest.com
I hope it will work on unrooted stock. I have some candidates.

Sent from my Clean ROM 3D

for tmo? it should the files are world readable - but it was quick and dirty code :) If not let me know and ill have er fixed up. Ill be cleaning up more in the next hours, just havta run up for a bit so i pushed my first checkpoint

XBpyK.png
 
Last edited:
  • Like
Reactions: guitardoc64

sgt. slaughter

Retired Forum Moderator
Jun 12, 2010
5,715
3,088
Raleigh
I made a quick and dirty android app for searching / sending profiles. I will be polishing it up more, anyone else feel free to jump in here to. The more grep strings or other locations these profiles are found will be added in here.


Right now it works only for tmobile flawlessly. But it will work for non rooted tmobile users, so if you feel comfortable send this to everyone on tmo you know to get the ball rolling. There is a compiled copy in bin\

https://github.com/TrevE/IQTool

cool deal.

you get a chance to see my response to you about md5sum you asked about?

just wanna know if i should send this file off or should i look for something else. lemme know if you want me to compare to anything you have on ur 3D. like i said mine is all stock kernel and all just on last OTA and rooted s-off.

....

That system/etc/iqprofile.pro might get updated on the e3d, were not sure yet how that works. Can you run busybox md5sum on that file?

I will post known hashes for stuff EFF has already (default profiles).

just to be sure i did this right, heres what i did:

adb shell
/system/xbin/busybox md5sum system/etc/iqprofile.pro

this output the following:
a6886135d2d1ea423d4edde389fe1794 system/etc/iqprofile.pro

and btw im running the stock OTA with full CIQ and stock kernel here.
 

Top Liked Posts

  • There are no posts matching your filters.
  • 9
    Hey guys, an EFF volunteer - Jered Wierzbicki - has created an application called IQIQ to decode Carrier IQ profiles back to XML. Their public git repo is in the above linked article. In a nutshell, Profiles contain what data is sent back at what interval and to where.

    We are now able to see sets of metrics called from phones. The problem is right now we only have default profiles. The EFF is looking to collect as many profiles from as many different devices as possible trying to find real metrics.


    Github Proof of concept code for a "profile scraper": https://github.com/TrevE/IQTool
    (right now it scans, but only works for automatic sending of archive.img from tmobile, but it should be for root or non root)
    If you can contribute go for it! It would help to have a "one click profile sender" out there for everyone.




    Possible Methods

    FINDING .PRO FILES
    We are not too sure yet where updated profiles get stored, so for the time being root users can run the following to search out all .pro files on your disk.

    Code:
    adb shell busybox find / -iname "*.pro"

    It will likely be some sort of IQProfile.pro, CIQProfile.pro, defaultprofile.pro. Once you locate it just:

    Code:
    adb pull /full/path/to/profile.pro .

    Other strategies, such as grepping for a characteristic string like " CONSTANT " across the whole filesystem, might also turn out to be useful. Use this thread to experiment with and improve such techniques!


    Waiters suggests a command reference:
    maybe something like this (which could take hours to run...)

    Code:
    grep -r "CONSTANT PROPID" /





    Getting archive.img from non-embedded CIQ
    We don't yet know if all profiles will be .pro files, or if they'll sometimes be embedded inside of other things. If you are on tmobile the profile information is potentially contained in an archive.img file. The file could be world readable, so you might not even need to be rooted.


    WARNING ABOUT ARCHIVE.IMG FORMATS:
    Please be warned that sensitive data could be in this archive.img file such as URLs, IMEI, SMS metadata, etc.. EFF will always do its best to keep archive.img files confidential, but please DO NOT send them if there may be any private information on the handset you are working with

    Example of where Tmobiles CIQ archive is:
    Code:
    adb pull /data/data/com.carrieriq.tmobile/app_iq_archive/archive.img .




    KNOWN STOCK MD5SUMS
    Were really trying to find some new profiles containing the pushed metrics, so to check if you are looking at a "stock" profile or not (this only applies to if you find a .pro file)

    Code:
    adb shell busybox md5sum /path/to/profile.pro

    If it matches one of these md5sums (number on the left) it is likely a known "stock" profile.
    Code:
    e37a4a8e3ea6d6aa4b7423a462541fa9  att-galaxy-s2-defaultProfile.pro
    2618eaa2e3310ec36e1b86f8b643c5fa  htc-amaze-tmob-defaultProfile.pro
    a6886135d2d1ea423d4edde389fe1794  htc-evo-sprint-iqprofile.pro
    2618eaa2e3310ec36e1b86f8b643c5fa  tmob-defaultProfile.pro


    SUBMITTING PROFILES

    If you would like to submit your profile or archive.img to EFF so CarrierIQ metrics on what operator can be better understood, please send that data off to iqiq@eff.org . It would be very helpful to them to include phone model and network it was pulled from as well.
    Thanks for all your help guys!
    2
    Portal article coming up. Thanks TrevE, and Jered and the EFF. We really appreciate everything you do.
    2
    maybe something like this (which could take hours to run...)

    Code:
    grep -r "CONSTANT PROPID" /
    2
    I made a quick and dirty android app for searching / sending profiles. I will be polishing it up more, anyone else feel free to jump in here to. The more grep strings or other locations these profiles are found will be added in here.


    Right now it works only for tmobile flawlessly. But it will work for non rooted tmobile users, so if you feel comfortable send this to everyone on tmo you know to get the ball rolling. There is a compiled copy in bin\

    https://github.com/TrevE/IQTool
    1
    I hope it will work on unrooted stock. I have some candidates.

    Sent from my Clean ROM 3D

    for tmo? it should the files are world readable - but it was quick and dirty code :) If not let me know and ill have er fixed up. Ill be cleaning up more in the next hours, just havta run up for a bit so i pushed my first checkpoint

    XBpyK.png