[DEV|APPv14] CIQ / HTC & Google Checkin / HTC loggers / Tell HTC Info & Removal

---

********LOGGING REVEAL***********


Over the past few weeks there has been alot about these "spyware" services running on our phones. I'm looking for dev input/a single place to document this all. I would also love for HTC to show us the real off switches for everything as they seem to make a big point about "opting out" lately - http://www.xda-developers.com/androi...ta-collection/ This is all obviously more than just tell HTC crash reporting.

The only way I have found to "opt-out" is to make changes below. HARDLY an on/off switch. You can see my analysis of stock logs getting written and processes running even though i "opted out" of everything officially possible here, here , and here



Other devs, please chime in with your findings I also have a more non technical writeup, a poll and visio diagram up in general if you just want a quick and dirty picture of whats going on here



3 reasons I think this is important:

1- Where did I "opt-in" to this? It would be entirely one thing if it was like location, where an agree message pops up, but as for most of these I havent seen anything of the sorts. I would not care in the least if it was apparent.

2- Why does htc/sprint/google whoever need to waste my mobile data (goes through my airave, so its my bandwidth) and CPU cycles/battery logging?

3- Who the hell is getting my data?

Below are the services, and i believe what goes to what.

Google Checkin - /system/app/GoogleServicesFramework.apk. reports to https://android.clients.google.com/checkin see analysis here
-Dropbox Services- I had 500+ files in it including full logcat/dmesg etc. Synced on google checkin.
-App Usage Stats- what apps you use, install, uninstall time used uploaded with dropbox

CIQ
HTC/Sprint, not sure whose responsible for this. User Behavior Logging Service may be related
-You can see what info the app seems capable of sending here hidden test functions detailed here


Tell HTC - Tell HTC reports App crashes to HTC automatically. Even if you shut this off in htcstetupwizard it still runs and logs data.


HTC Loggers - Seems to log certain kernel events. Has a hidden UI See http://forum.xda-developers.com/show...8&postcount=44


HTC Checkin - sends recovery logs (real cute), app usage stats, dmesg, mobile data usage etc. See here

-HTC App Usage Stats. I believe these get get sent out when you "checkin"

HTC App Usage Stats is triggered on the following:

1 = ("com.htc.feedback", "feedback_usageOpt")
2 = ("com.android.browser", "user_action")
3 = ("com.android.htccontacts", "contact_type")
4 = ("com.android.htccontacts", "group_info")
5 = ("com.android.mms", "message_send")
6 = ("com.android.mms", "message_receive")
7 = ("com.android.mms", "message_count")
8 = ("com.android.phone", "user_action")
9 = ("com.android.phone", "settings_quietRing")
10 = ("com.android.phone", "settings_pocketMode")
11 = ("com.android.phone", "settings_flipForSpeaker")
12 = ("com.android.phone", "edit_b4_call")
13 = ("com.futuredial", "transfer_data")
14 = ("com.htc.album", "storage")
15 = ("com.htc.android.htcime", "press_duration")
16 = ("com.htc.android.htcime", "UDB_words")
17 = ("com.htc.android.htcime", "special_correction")
18 = ("com.htc.android.htcime", "WCL_cnt")
19 = ("com.htc.android.htcime", "duration_SIP")
20 = ("com.htc.android.htcime", "lang_key")
21 = ("com.htc.android.htcime", "voice_key")
22 = ("com.htc.android.htcime", "set_CIME")
23 = ("com.htc.android.htcime", "duration_SIP")
24 = ("com.htc.android.htcime", "user_action")
25 = ("com.htc.android.htcime", "waiting_time")
26 = ("com.htc.android.htcime", "fuzzy_pinYin")
27 = ("com.htc.android.htcsetupwizard", "is_sysTimeChanged")
28 = ("com.htc.android.htcsetupwizard", "feedback_usageOpt")
29 = ("com.htc.android.mail", "mail_count")
30 = ("com.htc.android.mail", "update_schedule")
31 = ("com.htc.android.mail", "default_mailsize")
32 = ("com.htc.android.mail", "widget_clickcount")
33 = ("com.htc.android.mail", "EAS_success")
34 = ("com.htc.launcher", "layout")
35 = ("com.htc.launcher", "scene")
36 = ("com.htc.launcher", "user_action")
37 = ("com.htc.launcher", "app_launch")
38 = ("com.htc.launcher", "leap_view")
39 = ("com.htc.launcher", "skin_picker")
40 = ("com.htc.music", "storage")
41 = ("device_status", "battery")
42 = ("device_status", "battery_low")
43 = ("device_status", "bluetooth")
44 = ("download_manager", "download")
45 = ("system_server", "application_launch")
46 = ("system_server", "activity_launch_history")
47 = ("system_server", "MRU_click")
48 = ("system_server", "activity_tabCount")
49 = ("system_server", "tab_index")
50 = ("system_server", "app_uninstalled")
51 = ("system_server", "launcher_downloaded")

Testing / Removal App

I have also written an app to check these services easily. See all details for that here - http://forum.xda-developers.com/show...&postcount=110

Manual Removal

Part 1 - Framework.jar
Part 2 - Services.jar
Part 3 - Ramdisk/Kernel Source Misc Files

Framework.jar

---

Framework.jar -

/com/htc/utils/ulog/ULog.smali- this stops logging services from starting (removing init lines)

.method public static log(Lcom/htc/utils/ulog/ULogData;)V

    sget-object v1, Lcom/htc/utils/ulog/ULog;->sUserBehaviorLoggingService:Lcom/htc/utils/ulog/IUserBehaviorLoggingService;

    if-nez v1, :cond_2

  ORIGINAL LINE-  invoke-static {}, Lcom/htc/utils/ulog/ULog;->init()V
  MODDED LINE-  goto :goto_0

.method private static log(Ljava/lang/String;)V

    :cond_1
    sget-object v1, Lcom/htc/utils/ulog/ULog;->sUserBehaviorLoggingService:Lcom/htc/utils/ulog/IUserBehaviorLoggingService;

    if-nez v1, :cond_2

    ORIGINAL LINE-   invoke-static {}, Lcom/htc/utils/ulog/ULog;->init()V
  MODDED LINE-  goto :goto_0

.method public static logStat(Lcom/htc/utils/ulog/ULogStat;)V

    sget-object v1, Lcom/htc/utils/ulog/ULog;->sUserBehaviorLoggingService:Lcom/htc/utils/ulog/IUserBehaviorLoggingService;

    if-nez v1, :cond_2

   ORIGINAL LINE-  invoke-static {}, Lcom/htc/utils/ulog/ULog;->init()V
  MODDED LINE-  goto :goto_0

/com/htc/profileflag/TellHtcController.smali - disables auto reporting to HTC

.method static constructor <clinit>()V

    .locals 11

    const/16 v9, 0xc

    const/16 v6, -0x6b

   original line const/4 v8, 0x1
modded line  const/4 v8, 0x0

    sget-object v5, Lcom/htc/profileflag/TellHtcController;->EnabledTellHtcDevice:[B

    aget-byte v5, v5, v3

    if-ne v5, v6, :cond_4

 original line const/4 v0, 0x1
 modded line  const/4 v0, 0x0

.method public static isDisableErrorRport()Z

    .locals 1

     original line sget-boolean v0, Lcom/htc/profileflag/TellHtcController;->profile_force_disable_error_report:Z
    modded line   const/4 v0, 0x1

    return v0
.end method

.method public static isDisableUlog()Z

    .locals 1

   original line sget-boolean v0, Lcom/htc/profileflag/TellHtcController;->profile_force_disable_ulog:Z
    modded line   const/4 v0, 0x1
    return v0
.end method

.method public static isEnableAutoSend()Z

    .locals 1

    original line sget-boolean v0, Lcom/htc/profileflag/TellHtcController;->setting_enable_auto_send:Z
    modded line   const/4 v0, 0x0
    return v0
.end method

/com/htc/profileflag/HtcBuildFlag.smali Tells framework it was built without CIQ support

.field public static final HtcCIQFlag:Z = true change to false

/com/android/internal/app/IHtcAppUsageStatsService$Stub.smali - Disables HTC app usagestats service

.method public static asInterface(Landroid/os/IBinder;)Lcom/android/internal/app/IHtcAppUsageStatsService;

replace WHOLE old method with just following
  .locals 1

    const/4 p0, 0x0

    return-object p0

/com/android/internal/app/IUsageStats$Stub
.method public static asInterface(Landroid/os/IBinder;)Lcom/android/internal/app/IUsageStats;

replace WHOLE old method with just following
  .locals 1

    const/4 p0, 0x0

    return-object p0

/android/provider/htcCheckin.smali htc/google checkin disable

original
    sput-boolean v0, Landroid/provider/htcCheckin;->bCHECKIN:Z
    sput-boolean v0, Landroid/provider/htcCheckin;->bGOOGLE_CHECKIN:Z

modded
    sput-boolean v1, Landroid/provider/htcCheckin;->bCHECKIN:Z
    sput-boolean v1, Landroid/provider/htcCheckin;->bGOOGLE_CHECKIN:Z

/android/net/http/htchttpiqagent.smali - IQ agent uploader (i think)

.field public static final CIQVersion:Ljava/lang/String; = "1.1.1" change to "6.6.6" (anything > 3)
.field static final enableCIQ:Z = true change to false

    .locals 8

original    const/4 v2, 0x1
modded const/4 v2, 0x0

    const-wide/16 v4, 0x0

    const/4 v3, 0x0

    const-string v7, "EVDO"

    const-string v6, "1.1.1" anything > 3.0.0

    const-string v0, "1.1.1" anything > 3.0.0

    const-string v0, "1.1.1" anything > 3.0.0

.method public static getCIQFlag()Z
    .locals 1

original    const/4 v0, 0x1
modded  const/4 v0, 0x0

    return v0
.end method

/android/media/MediaCIQ
.method public static enableCIQ(Ljava/lang/String;)Z

    :cond_0
original line    const/4 v1, 0x1
modded line  const/4 v1, 0x0

    :goto_0
    return v1

    :cond_1
    const/4 v1, 0x0

    goto :goto_0

/com/android/internal/telephony/SMSDispatcher.smali

.field public static final COMPILE_OPTION_CIQ_SUPPORT:Z = true change to false

Services.jar -

/com/android/server/ulog:
Utils.smali-

.method static isTellHtcEnableAutoSend(Landroid/content/Context;)Z

    .locals 4

original:    const/4 v3, 0x1
modded:    const/4 v3, 0x0

.method static isUserProfilingSettingEnabled(Landroid/content/Context;)Z

    .locals 4

original:    const/4 v3, 0x1
modded:    const/4 v3, 0x0

UserBehaviorLoggingService.Smali

original:    const/4 v0, 0x1
modded:  const/4 v0, 0x0
    iput-boolean v0, p0, Lcom/android/server/ulog/UserBehaviorLoggingService;->mEnableHTCUBLog:Z

PolicyStore$DefaultPolicyLoader.smali- Defaults for database on what to log.

There are 10000s of changes I made, just search for "1" and change to "0" (for the most part) These are the trigger actions

There are also 2 IPs, change to 127.0.0.1

    const-string v2, "policy"

    const-string v3, "url"

    const-string v5, "175.41.155.212:2021"

    const-string v2, "log"

    const-string v3, "url"

    const-string v5, "175.41.164.137:8000"

/com/android/server/ServerThread.smali- Dropbox/Userbehavior Disable

    invoke-direct {v9, v6, v10}, Lcom/android/server/DropBoxManagerService;-><init>(Landroid/content/Context;Ljava/io/File;)V

original: invoke-static {v5, v9}, Landroid/os/ServiceManager;->addService(Ljava/lang/String;Landroid/os/IBinder;)V
modded:     invoke-static {v5, v9}, Landroid/os/ServiceManager;->deleteService(Ljava/lang/String;Landroid/os/IBinder;)V

    invoke-direct {v9, v6}, Lcom/android/server/ulog/UserBehaviorLoggingService;-><init>(Landroid/content/Context;)V

original:    invoke-static {v5, v9}, Landroid/os/ServiceManager;->addService(Ljava/lang/String;Landroid/os/IBinder;)V
 Modded:      invoke-static {v5, v9}, Landroid/os/ServiceManager;->deleteService(Ljava/lang/String;Landroid/os/IBinder;)V

/com/android/server/am/HtcAppUsageStatsService.smali - this is the dreaded HTC logging that has no off switch. This is the only way I could disable by replacing full methods with nulls.

remove all old junk and make these methods look exactly like this:
.method public notePackageRemoved(Ljava/lang/String;Ljava/lang/String;)V
    .locals 0

    return-void
.end method

.method public notePauseActivity(ILjava/lang/String;)V
    .locals 0

    return-void
.end method

.method public noteRecentTaskChange(Lcom/android/server/am/TaskRecord;)V
    .locals 0

    return-void
.end method

.method public noteResumeActivity(ILjava/lang/String;ZZ)V
    .locals 1

    return-void
.end method

.method public noteResumeActivity_pkg(ILjava/lang/String;Ljava/lang/String;ZZ)V
    .locals 1

    return-void
.end method

.method public publish(Landroid/content/Context;)V
    .locals 0

    return-void
.end method

.method public registerAppLaunchObserver(Lcom/android/internal/os/IHtcAppLaunchObserver;)V

    .locals 5
    .annotation system Ldalvik/annotation/Throws;
        value = {
            Landroid/os/RemoteException;
        }
    .end annotation

original line:     if-nez p1, :cond_0
modded line: if-eqz p1, :cond_0

Apps/Libs/Services:

/system/app/MyReportAgent.apk - tell htc
/system/app/HtcLoggers.apk - writes to /data/data/com.htc.loggers/. Has a checksu function. Analysis here
/system/app/HTCIQAgent.apk - IQ agent app. Analysis here
/system/app/CheckinProvider.apk - HTC Checkin. Analysis here
/system/bin/htcipcd - HTC IPC server. Analysis here
/system/bin/iqfd - CIQ frontend daemon. Analysis here  
/system/bin/iqd - CIQ backend daemon. Analysis here
/system/bin/androidvncserver - VNC remote screen display
/system/bin/usbnet - config for local vnc
/system/lib/libciq_client.so - ciq client lib
/system/lib/libciq_htc.so - ciq lib
/system/lib/libhtciqagent.so - ciq agent lib
/system/etc/iqprofile.pro - has a url for https://collector.iota.spcsdns.net:10003/collector/c
 
GSM/OTHER ROMS(For now)
/system/app/RamdumpEnabler.apk"
/sys/lib/libhtc_ramdump.so
/system/app/NetLogger.apk
/sys/lib/libhtc_netlogger.so - writes to /data/data/com.htc.android.netlogger/.  Uses TCPDUMP!??!?!?!??!?!?! (SEE WIRESHARK)

Build.prop

ril.iq.quickboot=1 - IQ start 

# Disable checkin/profiling services
ro.config.htc.nocheckin=1
ro.config.nocheckin=1
profiler.force_disable_err_rpt=1
profiler.force_disable_ulog=1


ALL OF THESE MIGHT NOT NEED TO BE REMOVED
id guess at crash/checkin being bad.  replace url with http://127.0.0.1  so you dont go to defaults.
# For FOTA setting (leave empty value to use default)
ro.htc.checkin.url = http://andchin.htc.com/android/checkin
ro.htc.checkin.crashurl = http://andchin.htc.com/android/crash
ro.htc.checkin.url_CN = http://andchin.htccomm.com.cn/android/checkin
ro.htc.checkin.crashurl_CN = http://andchin.htccomm.com.cn/android/crash
ro.htc.checkin.exmsg.url = http://fotamsg.htc.com/android/extra/
ro.htc.checkin.exmsg.url_CN = http://fotamsg.htccomm.com.cn/android/extra/
ro.htc.appupdate.url = http://apu-chin.htc.com/check-in/rws/and-app/update
ro.htc.appupdate.url_CN = http://apu-chin.htccomm.com.cn/check-in/rws/and-app/update
ro.htc.appupdate.exmsg.url = http://apu-msg.htc.com/extra-msg/rws/and-app/msg
ro.htc.appupdate.exmsg.url_CN = http://apu-msg.htccomm.com.cn/extra-msg/rws/and-app/msg

PLACES IN KERNEL/RAMDISK

PLEASE HAVE YOUR KERNEL DEV DISABLE STUFF IN THIS POST - http://forum.xda-developers.com/show...0&postcount=68

In ramdisk remove the following

#for CIQ ipc
mkdir /app-cache/ciq 0711 root system
mkdir /app-cache/ciq/socket 0777 root system

# HTC IPC server
service htcipcd /system/bin/htcipcd
    user root
    group root system

# CIQ backend daemon
service iqd /system/bin/iqd
    user root
    group root system

# CIQ frontend daemon
service iqfd /system/bin/iqfd
    user root
    group root system


service htc_ebdlogd /system/bin/htc_ebdlogd -s -k -P 7
    user root
    disabled
    oneshot
    ioprio idle 0

service htc_ebdlogd_rel /system/bin/htc_ebdlogd -s -k
    user root
    disabled
    oneshot
    ioprio idle 0


on property:ril.iq.quickboot=1
    start iqfd
    start iqd


# for vnc
service androidvncserver /system/bin/androidvncserver -a
    disabled
    oneshot

service usbnet /system/bin/usbnet on
    disabled
    oneshot

While your in ramdisk, you will want to change default.prop. Here we can hide our bootloader and unlock state from being reported to htc/sprint/google/whoever else. See - http://forum.xda-developers.com/show...&postcount=279 The following two should be set in default.prop:

ro.lb=unknown
ro.bootloader=unknown

DIRECTORIES/ FILES USED:

After making all above changes, remove anything in the below directories than reboot. Make sure nothing new is being added

/data/anr/
/data/data/com.android.htcprofile/
/cache/recovery/
/data/wimax/log/
/devlog
/data/system/usagestats
/data/system/appusagestats
/data/system/dropbox
/data/system/userbehavior.db
/data/system/userbehavior.xml
/app-cache/ciq/
/app-cache/iqserver
/data/misc/agent_htc/
/data/data/com.htc.loggers/

For safety you can chmod 400 or 000. Doing this alone without above framework changes will result in errors on write

How to tell Services running on phone

adb reboot
adb shell
logcat | grep Service

also make sure you dont see iq stuff here

adb shell
getprop | grep svc

Well may grab a spot in case I find some too.....


Sent from my PG86100 using Tapatalk

Thank you all for the work you have done already and for future works to come.

Awesome work! Thanks!

Great work man

Thanks guys, im passing out for now ill update more later. Im pretty sure everything to disable all the services are there now, ill get into the places its in ramdisk/etc later.

Hopefully it gets my point across to HTC how freeking hard it is to shut this off, and that we need a real optout? This stuff is hardcoded everywhere, especially Htc App Usage crap, its really not ment to be shut off. It was alot of work tracing these methods all through framework.

My main goal was trying to find that off switch to completely shut it down, that way it wouldn't be running in the background wasting CPU time. I haven't had a chance to look at the kernel yet but framework wise it looks like there are multiple flags that can disable portions of it. A lot of the stuff I found was already covered by TrevE so I'll do my best not to post anything redundant.

This is mainly for cIQ

Files Involved(incomplete)

Apks:
Browser.apk
Calculator.apk -- Front end?
HtcContacts.apk
HtcDialer.apk
HtcIQAgent.apk
IQRD.apk
SprintZoneNC.apk

Framework:
android.policy.jar
com.htc.framework.jar
framework.jar
services.jar

Binaries:
htcipcd
htc_inittest
htc_timezonetest
iqd
iqfd

Libraries:
libciq_client.so
libcig_htc.so
libhtciq_agent.so

To prevent metrics data from being sent (still logs though) getCIQFlag()z in framework.jar -> HTChttpIQagent.java needs to return False