[HOWTO] Fix CallerID matching issues in JB Phone and Messaging

C3C076 · May 7, 2013

How to fix CallerID matching issues in JellyBean Phone and Messaging apps

Introduction
This guide is for everyone who has problems with incorrect Caller ID
matching in Phone and Messaging that happens mainly on stock JellyBean ROMs.
This can be easily fixed in the source code, I know, but this guide is dedicated to devices
that have no source code available. That's why we will be dealing with some smali hacking.

For devs: It took me couple of hours to identify the problem and fix it,
so please, if you decide to use this how-to to fix stuff in your custom roms,
I expect you to give proper credits. Thanks in advance.

The guide does not cover things like deodexing/decompiling/compiling/odexing jars and apks.
It expects you already are familiar with these processes.
There are already many guides for those out there.

Symptoms
As you already know, operator often sends the full Caller ID including country prefix
when receiving a call. If your contacts are stored without country prefix (in local format),
Phone app is unable to match the number provided by the operator to one in the Contacts database.
Example: you have a contact defined with phone number 0902 xxx yyy.
Operator sends +421 902 xxx yyy - you get mismatch and no caller indicated while ringing.

In Messaging, when you send SMS to contact that has not country prefix, and receive response,
operator will most likely use full number with country prefix in a received SMS.
The messaging app is unable to associate arriving message to existing conversation because of
a number mismatch and thus creates a new conversation thread.

Cause
General
Contacts are stored within Contacts Provider. One of the attribute for contact is a "min_match"
attribute which stores a part of the whole number (from right to left) that has to be used when matching Caller ID to
contact. The length of the min_match attribute is defined in the telephony framework.
It is good practice to keep the MIN_MATCH length to max 7 (typically it takes 6 numbers from the right
+ 1 number from local prefix - for our example above min_match would be: 2xxxyyy
However, some official ROMs have the MIN_MATCH constant too high making it impossible to match
numbers stored in local format to full caller ID received from operator. (It was 11 in my case!!!)
(+421 902 xxx yyy != 0902 xxx yyy)

Apart from this, there is also one boolean constant that defines the method of caller ID comparation:

Code:

config_use_strict_phone_number_comparation

If this constant is set to "true" MIN_MATCH doesn't play role and always full numbers have to match to
correctly map caller to contact - this is something we don't want.

Messaging app specific
Apart from min_match thing I've alredy described, there's another part in Messaging that plays role.
Contacts in messaging app are cached in a local cache. Each cached contact is identified by a key, which
is typically a phone number. Again, there's a constant that defines how much characters from right to left
of the phone number have to be used as a key. If this constant is too big (10 in my case!!!), you will get mismatch when
Messaging app is trying to associate arriving SMS with existing conversation resulting in creating new thread.
It is good practice to keep this constant at the length of 5.

Solution
I) Check for and fix comparation method
Firstly, we must assure that strict checking is turned off.

1) Decompile framework-res.apk
2) Open res/values/bools.xml and search for config_use_strict_phone_number_comparation variable
3) Make sure it's false

Code:

<bool name="config_use_strict_phone_number_comparation">false</bool>

4) Recompile farmework-res.apk if you had to change the variable

II) Check for and fix MIN_MATCH constant in telephony framework
1) Decompile framework.jar
2) Open smali/android/telephony/PhoneNumberUtils.smali
3) Search for MIN_MATCH constant and set it to 0x7

Code:

.field static final MIN_MATCH:I = 0x7 (was 0xb in my case)

4) Since in smali, constant names are not used but they are translated to explicit values, we have
to search for all occurences where old MIN_MATCH constant is used (0xb in my case) and change the values to 0x7
a) one is in "compareLoosly" method:

Code:

.local v7, minMatchLen:I
const/16 v7, 0x7 (was 0xb in my case)

b) another one in "toCallerIDMinMatch" method:

Code:

.local v0, minMatchLen:I
const/16 v0, 0x7 (was 0xb in my case)

5) Save the file and recompile framework.jar

III) Fix messaging app
1) Decompile Mms.apk
2) Open smali/com/android/mms/data/Contact$ContactsCache.smali
3) Search for STATIC_KEY_BUFFER_MAXIMUM_LENGTH and set it to 0x5

Code:

.field static final STATIC_KEY_BUFFER_MAXIMUM_LENGTH:I = 0x5 (was 0xa in my case)

4) Search for all occurences where old STATIC_KEY_BUFFER_MAXIMUM_LENGTH constant is used (0xa in my case) and change the values to 0x5
a) one is in "key" method

Code:

const/16 v2, 0x5 (was 0xa in my case)

b) another one in "getKey" method (it is possible there is no such method, then ignore it)

Code:

const/16 v4, 0x5 (was 0xa in my case)

c) another one in "internalGet" method

Code:

const/16 v10, 0x5 (was 0xa in my case)

5) Save the file and recompile Mms.apk

IMPORTANT:
After pushing your new files to phone it is necessary to re-add all of the contacts for fix to take effect.
This is because your contacts are still stored in a database with wrong min_match number.
It needs to be re-initialized according to new MIN_MATCH constant we have modified.
The fastest way is to go to Settings/Apps, locate Contacts Storage, open it and Clear data.
(this will delete all contacts from phone so make sure you have a backup).
In case you have your contacts synced to Google account, no backup is necessary.
Your contacts will be synced a while after you clear data for Contacts Storage.

For Messaging it is necessary to delete all the old threads that are incorrectly splitted.
All new conversations created after fix has been applied will be grouped OK.

romuleta · Jul 26, 2013

Thanks a lot man!! My wife's Lenovo A820 is showing the caller id now.

frapedas · Aug 14, 2013

C3C076 said:
How to fix CallerID matching issues in JellyBean Phone and Messaging apps

For Messaging it is necessary to delete all the old threads that are incorrectly splitted.
All new conversations created after fix has been applied will be grouped OK.

Help me with this please, im trying to fix this on a cheap mocordroid, i edited the smali PhoneNumberUtils all the fuctions that used 0xb for loosycompare, and recompilted just fine, i even checked the contacts2.db and indeed now the min_match table cointains rows of 7' numbers, but the problem persists to be there, only 11 n. and more are getting recognized as the same contact.. both at phone boot and mms, i even tried apps like go sms and again.. the same happens.. so where to look now?

I decompiled ContactProvider.apk and i have look into it for over than 100 times through all its lines and i can't find anything that will make me suspect it is its fault.. the queries seems fine and i can't find any if' conditions that whould bother the database queries of 'min_match=x' so.. whats left now to look into? its really drivers me crazy

C3C076 · Aug 14, 2013

Did you delete and re-add all of your contacts after the change?

frapedas · Aug 14, 2013

C3C076 said:
Did you delete and re-add all of your contacts after the change?

Of course, i even did wipe, and readed everything, again only after 11 nums it works, i even decompiled again PhoneNumberUtils from framework, and all the 0xb values are now 0x7, i also opened contacts2.db and the min_max rows contains also only 7 numbers, which is weird, because the database is ok.. is 7, but phonebook + sms works only with 11 or more.. its driving me crazy

C3C076 · Aug 15, 2013

What about config_use_strict_phone_number_comparation
Global setting?

frapedas · Aug 15, 2013

C3C076 said:
What about config_use_strict_phone_number_comparation
Global setting?

Ok guys.. i toke it a bit further.. its not an issue of my phone is an issue of Android 2.3.3 default sources!!!

Firstly to make sure isn't the framework, i downloaded the android source code, i extracted the class PhoneNumberUtils.Java and i didn't had to edit it because it was already 7 .. then i used dex2jar and converted framework (classes.dex) to framework.jar so i can link it to the compiler and i can compile from the start the java file, the class was compiled ok without any problems, i re-placed it back to the jar, i converted it back to dex, etc etc and placed it to my phone...guess what?????

SAME BEHAVIOUR!!!! only up to 11..

Then i think.. WTF this is from the original android sourcecode compiled... and i installed android SDK 2.3.3 and i run an emulator from the virtual device manager... guess what???

ORIGINAL android 2.3.3 as it is provided from Google, it parses only contacts up to 11 numbers, even its MIN_MATCH is 7!! both in sms app again and in phone app...

This is a good thing and a bad thing, it is a good because we can download the whole source code including the one of apks and spot wtf is doing this.. is a bad because still didn't found a fix for 2.3.3

After some researching ive started suspecting that TelephonyProperties.PROPERTY_ICC_OPERATOR_ISO_COUNTRY or TelephonyProperties.PROPERTY_IDP_STRING or TelephonyProperties.PROPERTY_OPERATOR_ISO_COUNTRY since those strings can be defined in build.prop file.. and i really cant find any information about these settings on google or anywere.. i mean whats ICC ? whats IDP string? what they mean?

So if anybody of you want to see that behaviour as a proof and you have android SDK just run a virtual emulator of 2.3.3 add a contact of +xxNNNNNNNNNN ( 2 country code + 10 numbers) and just try to dial only the 10 numbers without the code.. BOOM.. nothing happens! even its min_match is 7 and its compireloosely on its settings!

if you manage to fix this at least on your virtual emulator i will be really greateful.. im still trying...

frapedas · Aug 15, 2013

Finally i FIXED IT.. thanks to my memory sundenly i had a flash back about reading having to change sqlite.so lib... the first time i readed it i said... bull...t thought its not possible a module for handling sql queries to have such kind of limits..

after trying everything i gave it a shot, trying from random roms sqlite.so's.. and with one that i found its ok.!!

just for the record, i replaced sqlite.so and sqlite_jini.so from. pfff i had enough of this

CSCicus · Aug 17, 2013

C3C076 said:
How to fix CallerID matching issues in JellyBean Phone and Messaging apps

Introduction
This guide is for everyone who has problems with incorrect Caller ID
matching in Phone and Messaging that happens mainly on stock JellyBean ROMs.
This can be easily fixed in the source code, I know, but this guide is dedicated to devices
that have no source code available. That's why we will be dealing with some smali hacking.

For devs: It took me couple of hours to identify the problem and fix it,
so please, if you decide to use this how-to to fix stuff in your custom roms,
I expect you to give proper credits. Thanks in advance.

The guide does not cover things like deodexing/decompiling/compiling/odexing jars and apks.
It expects you already are familiar with these processes.
There are already many guides for those out there.

Symptoms
As you already know, operator often sends the full Caller ID including country prefix
when receiving a call. If your contacts are stored without country prefix (in local format),
Phone app is unable to match the number provided by the operator to one in the Contacts database.
Example: you have a contact defined with phone number 0902 xxx yyy.
Operator sends +421 902 xxx yyy - you get mismatch and no caller indicated while ringing.

In Messaging, when you send SMS to contact that has not country prefix, and receive response,
operator will most likely use full number with country prefix in a received SMS.
The messaging app is unable to associate arriving message to existing conversation because of
a number mismatch and thus creates a new conversation thread.

Cause
General
Contacts are stored within Contacts Provider. One of the attribute for contact is a "min_match"
attribute which stores a part of the whole number (from right to left) that has to be used when matching Caller ID to
contact. The length of the min_match attribute is defined in the telephony framework.
It is good practice to keep the MIN_MATCH length to max 7 (typically it takes 6 numbers from the right
+ 1 number from local prefix - for our example above min_match would be: 2xxxyyy
However, some official ROMs have the MIN_MATCH constant too high making it impossible to match
numbers stored in local format to full caller ID received from operator. (It was 11 in my case!!!)
(+421 902 xxx yyy != 0902 xxx yyy)

Apart from this, there is also one boolean constant that defines the method of caller ID comparation:

Code:

config_use_strict_phone_number_comparation

If this constant is set to "true" MIN_MATCH doesn't play role and always full numbers have to match to
correctly map caller to contact - this is something we don't want.

Messaging app specific
Apart from min_match thing I've alredy described, there's another part in Messaging that plays role.
Contacts in messaging app are cached in a local cache. Each cached contact is identified by a key, which
is typically a phone number. Again, there's a constant that defines how much characters from right to left
of the phone number have to be used as a key. If this constant is too big (10 in my case!!!), you will get mismatch when
Messaging app is trying to associate arriving SMS with existing conversation resulting in creating new thread.
It is good practice to keep this constant at the length of 5.

Solution
I) Check for and fix comparation method
Firstly, we must assure that strict checking is turned off.

1) Decompile framework-res.apk
2) Open res/values/bools.xml and search for config_use_strict_phone_number_comparation variable
3) Make sure it's false

Code:

<bool name="config_use_strict_phone_number_comparation">false</bool>

4) Recompile farmework-res.apk if you had to change the variable

II) Check for and fix MIN_MATCH constant in telephony framework
1) Decompile framework.jar
2) Open smali/android/telephony/PhoneNumberUtils.smali
3) Search for MIN_MATCH constant and set it to 0x7

Code:

.field static final MIN_MATCH:I = 0x7 (was 0xb in my case)

4) Since in smali, constant names are not used but they are translated to explicit values, we have
to search for all occurences where old MIN_MATCH constant is used (0xb in my case) and change the values to 0x7
a) one is in "compareLoosly" method:

Code:

.local v7, minMatchLen:I const/16 v7, 0x7 (was 0xb in my case)

b) another one in "toCallerIDMinMatch" method:

Code:

.local v0, minMatchLen:I const/16 v0, 0x7 (was 0xb in my case)

5) Save the file and recompile framework.jar

III) Fix messaging app
1) Decompile Mms.apk
2) Open smali/com/android/mms/data/Contact$ContactsCache.smali
3) Search for STATIC_KEY_BUFFER_MAXIMUM_LENGTH and set it to 0x5

Code:

.field static final STATIC_KEY_BUFFER_MAXIMUM_LENGTH:I = 0x5 (was 0xa in my case)

4) Search for all occurences where old STATIC_KEY_BUFFER_MAXIMUM_LENGTH constant is used (0xa in my case) and change the values to 0x5
a) one is in "key" method

Code:

const/16 v2, 0x5 (was 0xa in my case)

b) another one in "getKey" method (it is possible there is no such method, then ignore it)

Code:

const/16 v4, 0x5 (was 0xa in my case)

c) another one in "internalGet" method

Code:

const/16 v10, 0x5 (was 0xa in my case)

5) Save the file and recompile Mms.apk

IMPORTANT:
After pushing your new files to phone it is necessary to re-add all of the contacts for fix to take effect.
This is because your contacts are still stored in a database with wrong min_match number.
It needs to be re-initialized according to new MIN_MATCH constant we have modified.
The fastest way is to go to Settings/Apps, locate Contacts Storage, open it and Clear data.
(this will delete all contacts from phone so make sure you have a backup).
In case you have your contacts synced to Google account, no backup is necessary.
Your contacts will be synced a while after you clear data for Contacts Storage.

For Messaging it is necessary to delete all the old threads that are incorrectly splitted.
All new conversations created after fix has been applied will be grouped OK.

Super Vdaka za vyvetlenie a navod.

eran.langa · Oct 26, 2013

help with decompile Mms.apk

C3C076 said:
How to fix CallerID matching issues in JellyBean Phone and Messaging apps

III) Fix messaging app
1) Decompile Mms.apk
2) Open smali/com/android/mms/data/Contact$ContactsCache.smali
3) Search for STATIC_KEY_BUFFER_MAXIMUM_LENGTH and set it to 0x5

Code:

.field static final STATIC_KEY_BUFFER_MAXIMUM_LENGTH:I = 0x5 (was 0xa in my case)

4) Search for all occurences where old STATIC_KEY_BUFFER_MAXIMUM_LENGTH constant is used (0xa in my case) and change the values to 0x5
a) one is in "key" method

Code:

const/16 v2, 0x5 (was 0xa in my case)

b) another one in "getKey" method (it is possible there is no such method, then ignore it)

Code:

const/16 v4, 0x5 (was 0xa in my case)

c) another one in "internalGet" method

Code:

const/16 v10, 0x5 (was 0xa in my case)

5) Save the file and recompile Mms.apk

IMPORTANT:
After pushing your new files to phone it is necessary to re-add all of the contacts for fix to take effect.
This is because your contacts are still stored in a database with wrong min_match number.
It needs to be re-initialized according to new MIN_MATCH constant we have modified.
The fastest way is to go to Settings/Apps, locate Contacts Storage, open it and Clear data.
(this will delete all contacts from phone so make sure you have a backup).
In case you have your contacts synced to Google account, no backup is necessary.
Your contacts will be synced a while after you clear data for Contacts Storage.

For Messaging it is necessary to delete all the old threads that are incorrectly splitted.
All new conversations created after fix has been applied will be grouped OK.

I don't know how to do this. can someone do for me the fixing of the Mms.apk so it will check for match just 8 characteristic (8 no.) please?
I attached the Mms.apk file.
that will solve the problem for all HUAWEI phone users.:laugh:

adeelraj · Oct 27, 2013

we need deodexed jar file for editing this.

felagund · Oct 27, 2013

Hi,

do you know by any chance if I should be looking in the phone apk, contacts provider, etc. for the following problem.

When the SIM card is not registered on network T9 dialing works ok and I can search for John in the name "John Smith" but when the card is registered I suddenly cannot write/search for more then 2 characters ending with a search for "jo"

Usually after the second number pressed with the sim card on there is a slash (probably to distinguish city /geographic code as most codes in my country are 2 digits long). This happens only when SIM card is registered to network.

I'm willing to deodex/decompile phone, mms, contacts provider, etc. but I don't really know which file is the one I need to fix this.

It's on a custom Lenovo A820 4.2 ROM ported from ZTE v987 and I'd like to sort this out and then share it with the developer

Please tell me if there is a more-appropriate topic where I should ask the question as I thing it's close enough to the topic in this thread.

Cheers!

eran.langa · Oct 27, 2013

adeelraj said:
we need deodexed jar file for editing this.

to who did you answer? for my question?
what is deodexed jar?

adeelraj · Oct 28, 2013

^
What do u mean

Sent from my IM-A850K using Tapatalk 2

bryan_mmx · Nov 28, 2013

is this applicable to JB 4.3? I'm not seeing the mms.apk in system/app folder to edit

thienhoa · Feb 24, 2014

Thank so much

Ouzo · May 2, 2014

C3C076 - Firstly I just wanted to say thanks for so comprehensively and succinctly explaining the nature of the problem in your opening post. The part about SMS messages being split into separate conversations was particularly useful as it matches exactly what is happening with the Jiayu G4S that I picked up a few days ago. So far I haven't come across this explanation anywhere else.

Unfortunately I am a complete noob at deodexing/decompiling/recompiling etc so I am currently trying to learn everything I need to. I've never really gone beyond flashing ROMs, rooting and changing recoveries, not to mention the fact that every ROM I've ever worked with has been deodexed. So far I've only managed to confirm that <bool name="config_use_strict_phone_number_comparation"> is set to false after decompiling framework-res.apk.

For some reason, possibly my inexperience and ignorance, I'm having a hard time working with framework.jar (even deodexing). After a few hours of trying to make progress I decided it might be easier to install Xposed and then your GravityBox mod but I soon realised I can't do this. Whilst looking in the Xposed thread I noticed a reference to the fact that it would not work on a bunch of Chinese devices because a different compiler had been used in their respective ROMs (hence Lewa OS I guess). For example, in my \System\Framework folder, in addition to framework.jar and framework.odex, I also have framework.jar.jex. Does that match the configuration of the stock ROM on your old W8? This discovery started to make me wonder if this is the reason I'm having trouble working with the files that I need to. Any pointers you or anyone else could give me so that I can actually get to the stage where I can start seeing and changing min_match values would be greatly appreciated.

As an aside, does anyone know why adding “ro.phone.min_match=7” to build.prop doesn't work on some devices (even after clearing contacts and re-importing them)?

C3C076 · May 2, 2014

Hi.

I made this guide prior to starting developing GravityBox (which already includes this fix).
Yes, I noticed some users within Xposed framework thread mentioning that strange "jex" format which prevents Xposed framework from working at all.
Couple of days ago, somebody posted info that it is possible to deodex such ROMs and make xposed framework work for them.
Unfortunately, I have no experience with such format so I cannot give you the necessary info how to decompile/change/recompile stuff or how to deodex. You might try to search the xposed framework thread and get in touch with one of the users that managed to get xposed framework working on ROM with jex files.

Ouzo · May 2, 2014

C3C076 said:
Hi.

I made this guide prior to starting developing GravityBox (which already includes this fix).
Yes, I noticed some users within Xposed framework thread mentioning that strange "jex" format which prevents Xposed framework from working at all.
Couple of days ago, somebody posted info that it is possible to deodex such ROMs and make xposed framework work for them.
Unfortunately, I have no experience with such format so I cannot give you the necessary info how to decompile/change/recompile stuff or how to deodex. You might try to search the xposed framework thread and get in touch with one of the users that managed to get xposed framework working on ROM with jex files.

Awesome, thanks. I'll spend more time in the Xposed thread and see what information I can glean.

Also, sorry, I arrived at this thread via a Google search result and didn't notice that it is for developers only otherwise I wouldn't have posted at all. I should know better by now - oops!

Man I wish I'd just bought the W8 last year when I was toying with the idea. So far I've had two defective iOcean X7S in the last month, both with bizarre touchscreen problems (other wise a lovely device) and now the G4S which has bleeding backlight and a few firmware bugs (and also lacks performance in the GPU department, which is odd given that it is pushing around half the pixels that the X7s did). It really does bring home how important it is to buy a device with good developer support from the community if you only 'dabble' with other people's work like me and aren't a developer yourself. LOL. On the other hand, a smartphone for £123 won't have to last me as long as a 'flagship' device costing me £550!

newbie_vlc · Jun 20, 2014

C3C076 said:
Hi.

I made this guide prior to starting developing GravityBox (which already includes this fix).
Yes, I noticed some users within Xposed framework thread mentioning that strange "jex" format which prevents Xposed framework from working at all.
Couple of days ago, somebody posted info that it is possible to deodex such ROMs and make xposed framework work for them.
Unfortunately, I have no experience with such format so I cannot give you the necessary info how to decompile/change/recompile stuff or how to deodex. You might try to search the xposed framework thread and get in touch with one of the users that managed to get xposed framework working on ROM with jex files.

If I try to use GravityBox to solve the sms problem, I have to delete contacs and resync it?

Thank you

[HOWTO] Fix CallerID matching issues in JB Phone and Messaging

Inactive Recognized Contributor

Senior Member

Senior Member

Inactive Recognized Contributor

Senior Member

Inactive Recognized Contributor

Senior Member

Senior Member

Senior Member

Senior Member

Attachments

Senior Member

Senior Member

Senior Member

Senior Member

Senior Member

Member

Senior Member

Inactive Recognized Contributor

Senior Member

Senior Member

Similar threads

Top Liked Posts