[Q] SMS time sent vs. received

[Brianakin]

I give up. I have searched here and elsewhere for a solution that works for me but nothing does. I am running a custom TW ROM (Dandroid 3.9 for JB 4.1.2) on Verizon firmware VRBMF1, so the fix for JPX/JPY firmware won't work for me. Other mods I have seen are for different firmwares also, and Wanam Xposed only works from JB 4.2+. I have also tried 3rd party apps that supposedly fix this, but SMS Sent Time Fix does not work, and SMS Sent Time does not actually fix the timestamp on the message; it only appends the actual sent time to the end of the body of the message which I don't prefer.

I did find a modded SecMms.apk with a fix baked in which does successfully add the actual sent time to the 'date_sent' field in the mmssms.db database; however this still does not accomplish the task of substituting the time sent in the timestamp that appears above each message. Also knowing that the SMS Sent Time app can locate the actual date/time sent gives me the belief that there has to be a way to get the timestamp in my conversations to use this value. Technically it wouldn't be the time I received it, but in cases where I'm in airplane mode for hours - as happens frequently - seeing that I received a text while my phone was off and knowing the time it was sent is more important that using a true time of receipt.

If I thought it was a limitation with JB 4.1.2 I would bite the bullet and update either to 4.3 or even to KitKat, but I like to think that it's possible yet the developers that do all this amazing work haven't found that Verizon SCH-i535 phone running TouchWiz Jellybean 4.1.2 to test on.

So does anyone have any ideas as to what I can try next?

 

[androcheck]

Hi!

I'm the developer of "SMS Sent Time". I haven't been active here on xda for some time now but I got noticed about your thread via Google Alerts and I thought I will use this occasion to bring some light into the overall situation:

Sorry, the post got much longer than intended, but there was so much to explain...


How SMS receiving works in general:
Your mobile network sends you a PDU (protocol data unit) packet containing a SMS message. This PDU format also contains a timestamp field which is filled by the SMSC server of the SMS sender. In most of the times this timestamp == sent time (the time when the SMSC of the sender's mobile network received the message by the sender's phone). However some obscure mobile network providers around the world seem to insert a wrong time into this field (mostly because of wrong set timezone settings, UTC time vs. local time, applying wrong timezone values, etc., you can imagine.. ).


How SMS are received/stored in Android
In Android there exists a system wide database where the received SMS messages get stored into. When a SMS arrives, a system service parses the raw PDU packet it received from the modem part of your phone and stores the SMS data into the system database. The user then can use any SMS application he likes (the default one, GoSMS, HandcentSMS, etc..) to display the messages from this global system SMS database. All SMS apps access the same underlying Android system SMS database.


The problem:
When implementing this SMS database and the receiving service Google made 2 decisions:

• they decided that the timestamp from the provider might not be reliable and
• they initially only included ONE "date" field in the system database (you can see the structure of this database table in old Android 2.2 here: Android 2.2 SMS database structure).

As a result we ended up having only one time field for a SMS and the SMS receiver service filled this time field with the current time when it receives the SMS (because it doesn't trust the SMSC timestamp).


What SMS Sent Time does:
Given this situation (and not liking it) I realized a few years ago that it is possible for any application in Android (of course only with the according permissions) to register for incoming SMS Intents (simply register a rceiver for "android.provider.Telephony.SMS_RECEIVED" in the app). When doing this the Android system informs the app on any incoming SMS and also lets it access the data from the raw PDU packet received from the network provider (more or less).

So in this moment (when receiving the SMS) the information about the SMSC timestamp is still available!! This was all I needed to build the SMS Sent Time application. But first I had some troubles when modifying the SMS database, mostly because of the fact, that my app first needs to wait until the system service has stored the SMS into the database and then my app has to reliably find the very same sms in the database. But when the message is found the app can modify it (prior to Android 4.4). So the app also could easily change the date field (instead appending it to the text). I was thinking about doing this, but then I got unsure that this might mess up the sort order in some SMS apps when I receive some SMS with a really completly wrong timestamp. So - just to be on the safe side - I decided to append it to the text (which didn't bother me personally).

I also have to say that at this point in time I had no intentions to make this application ever public. This was just meant as a quick hack for myself. Only some time later when I realized that a lot of more people are facing the same problem I decided to publish it. But it still was and is a quick hack for myself. I have not the resources (mostly time) to invest more work. I develop on Android only in my free time and never made a single cent with this app (no donations, no ads) but spent over the years hundreds of hours (including answering a lot of users' mails). So I was glad when people started asking me for the source code (it's public on GitHub: SMS Sent Time on Github) hoping different/improved solutions might arise...


What Google learned about this:
There existed a loooong time bug-report on exact this issue (sorry cannot find the link at the moment) where a lot of users where complaining about exactly this problem (sent time != received time). So Google finally listened to its users and changed the format of the SMS system database beginning Android 4 (AFAIK). You can see it here, if we look at the same database code as above in Version 4.0.3: Android 4.0.3 SMS database structure you will notice that now there is an additional field "date_sent" in the database table! Hooray! :victory: Finally!

Unfortunately the Android world is very fragmented and every phone manufacturer seems to go its own ways. So not all phones with Android >4.0 did implement this change or even if they did inherit this part of the Android core system they often did not update their SMS applications!

The same applies for 3rd party SMS apps. For example: I have a Nexus 4 running a cyanogen-based 4.4.1 and I can confirm that in my system database both date fields are filled correctly. So the sent time is definitely there! But neither "Go SMS Pro" nor "Handcent SMS" seem to display this second field (please correct me if I am wrong, I just made a quick test, I normally don't use these apps). Only the original AOSP SMS app and Google Hangout (when used as SMS app) seem to display both times (sent and received) in message details.


New limitations with Android KitKat (4.4):
Additionally another change came along the road when Google introduced Android 4.4 (KitKat). Suddenly a lot of users reported that SMS Sent Time stopped working for them. The background: Starting with Android 4.4 Google introduced the concept of a "default SMS app". The user has to choose, which app should be the default application for handling incoming SMS on the phone. While other SMS apps (for example like my SMS Sent Time) still are able to receive incoming SMS (and read the sent time) they are no longer allowed to modify the content of the system's SMS database (even with the correct permissions in place). The database update silently fails. Sad enough this is a fact and there is nothing I can do in my app to change this. It's a security feature of Android which I cannot change. (And the reason why I am writing a lot of feedback mails to users lately.)

Workaround:
But at least there is a known "workaround" to overcome this new KitKat limitation (altough it seems that Google decided to remove this possibility with the update from 4.4.1 to 4.4.2). In Android 4.4 there exists a hidden settings screen ("AppOps") which allows you to grant or deny specific permissions to applications. You cannot open this settings screen directly from the system settings, but there are some apps in the Play Store which do nothing else, than just opening this (already existing but hidden) settings screen. For example, I used the app App Ops Starter. This simple app needs no permissions and does nothing else then opening Android's hidden "app ops" settings screen. Once opened, on the tab "Messaging", you see a list with all your installed apps which have sms-permissions. If you open the settings page for one specific app you can grant the permission "Write SMS/MMS" again and the system will behave again like it did before Android 4.4.



So, long story short:
It's been a long time with this problem in the Android world. Hopefully workarounds like my app should not be needed in future (when all SMS apps make use of the "date_sent" field in the SMS database).

Until then it would be possible to modify the SMS Sent Time app to overwrite the datestamp field in the database (instead appending the time to the message text). Any develper should feel free to take the source code and modify it accordingly.


[edit]
corrected spelling
[/edit]

 

[Brianakin]

Now THAT is a response! Thanks so much for taking the time to explain in vivid detail (and I actually understands more than half of it!). I do recall the Google thread that dates back to 2009 and kept running across it in my searches. I also saw references to "App Ops" along the way and the suggestion that even that workaround fails starting with 4.4.2. So yeah there's a lot to digest here but I suppose the 3 takeaways for me are:

1) It is possible for 3rd party messaging apps like Handcent and Go SMS to implement the sent time either in message details or in the timestamp above the message.
2) That hope vanishes with KK 4.4.2
2) That hope could be restored if Android again revises its SMS database structure or the message handling process to capture and display the time the SMS was actually sent

Still the issue of different time zones can muddy the waters so it may be that a perfect solution does not exist, at least without a lot of extra effort to identify sender location, compare to receiver's location, look up a time zone offset value somewhere, etc. etc.

But at least your explanation clears up a LOT of confusion for me so now I know the sordid history behind it all. And I will say that though I don't prefer to see a timestamp at the end of my messages, it is better than not being able to get the real sent time at all.

Thanks again for making your "quick hack" available to us and for all the other support you've given the user community on this nagging issue!

Hi!

I'm the developer of "SMS Sent Time". I haven't been active here on xda for some time now but I got noticed about your thread via Google Alerts and I thought I will use this occasion to bring some light into the overall situation:

Sorry, the post got much longer than intended, but there was so much to explain...


How SMS receiving works in general:
Your mobile network sends you a PDU (protocol data unit) packet containing a SMS message. This PDU format also contains a timestamp field which is filled by the SMSC server of the SMS sender. In most of the times this timestamp == sent time (the time when the SMSC of the sender's mobile network received the message by the sender's phone). However some obscure mobile network providers around the world seem to insert a wrong time into this field (mostly because of wrong set timezone settings, UTC time vs. local time, applying wrong timezone values, etc., you can imagine.. ).


How SMS are received/stored in Android
In Android there exists a system wide database where the received SMS messages get stored into. When a SMS arrives, a system service parses the raw PDU packet it received from the modem part of your phone and stores the SMS data into the system database. The user then can use any SMS application he likes (the default one, GoSMS, HandcentSMS, etc..) to display the messages from this global system SMS database. All SMS apps access the same underlying Android system SMS database.


The problem:
When implementing this SMS database and the receiving service Google made 2 decisions:

• they decided that the timestamp from the provider might not be reliable and
• they initially only included ONE "date" field in the system database (you can see the structure of this database table in old Android 2.2 here: Android 2.2 SMS database structure).

As a result we ended up having only one time field for a SMS and the SMS receiver service filled this time field with the current time when it receives the SMS (because it doesn't trust the SMSC timestamp).


What SMS Sent Time does:
Given this situation (and not liking it) I realized a few years ago that it is possible for any application in Android (of course only with the according permissions) to register for incoming SMS Intents (simply register a rceiver for "android.provider.Telephony.SMS_RECEIVED" in the app). When doing this the Android system informs the app on any incoming SMS and also lets it access the data from the raw PDU packet received from the network provider (more or less).

So in this moment (when receiving the SMS) the information about the SMSC timestamp is still available!! This was all I needed to build the SMS Sent Time application. But first I had some troubles when modifying the SMS database, mostly because of the fact, that my app first needs to wait until the system service has stored the SMS into the database and then my app has to reliably find the very same sms in the database. But when the message is found the app can modify it (prior to Android 4.4). So the app also could easily change the date field (instead appending it to the text). I was thinking about doing this, but then I got unsure that this might mess up the sort order in some SMS apps when I receive some SMS with a really completly wrong timestamp. So - just to be on the safe side - I decided to append it to the text (which didn't bother me).

I also have to say that at this point in time I had no intentions to make this application ever public. This was just meant as a quick hack for myself. Only some time later when I realized that a lot of more people are facing the same problem I decided to publish it. But it still was and is a quick hack for myself. I have not the resources (mostly time) to invest more work. I develop on Android only in my free time and never made a single cent with this app (no donations, no ads) but spent over the years hundreds of hours (including answering a lot of users' mails). So I was glad when people started asking me for the source code (it's public on GitHub: SMS Sent Time on Github) hoping different/improved solutions might arise...


What Google learned about this:
There existed a loooong time bug-report on exact this issue (sorry cannot find the link at the moment) where a lot of users where complaining about exactly this problem (sent time != received time). So Google finally listened to its users and changed the format of the SMS system database beginning Android 4 (AFAIK). You can see it here, if we look at the same database code as above in Version 4.0.3: Android 4.0.3 SMS database structure you will notice that now there is an additional field "date_sent" in the database table! Hooray! :victory: Finally!

Unfortunately the Android world is very fragmented and every phone manufacturer seems to go its own ways. So not all phones with Android >4.0 did implement this change or even if they did inherit this part of the Android core system they often did not update their SMS applications!

The same applies for 3rd party SMS apps. For example: I have a Nexus 4 running a cyanogen-based 4.4.1 and I can confirm that in my system database both date fields are filled correctly. So the sent time is definitely there! But neither "Go SMS Pro" nor "Handcent SMS" seem to display this second field (please correct me if I am wrong, I just made a quick test, I normally don't use these apps). Only the original AOSP SMS app and Google Hangout (when used as SMS app) seem to display both times (sent and received) in message details.


New limitations with Android KitKat (4.4):
Additionally another change came along the road when Google introduced Android 4.4 (KitKat). Suddenly a lot of users reported that SMS Sent Time stopped working for them. The background: Starting with Android 4.4 Google introduced the concept of a "default SMS app". The user has to choose, which app should be the default application for handling incoming SMS on the phone. While other SMS apps (for example like my SMS Sent Time) still are able to receive incoming SMS (and read the sent time) they are no longer allowed to modify the content of the system's SMS database (even with the correct permissions in place). The database update silently fails. Sad enough this is a fact and there is nothing I can do in my app to change this. It's a security feature of Android which I cannot change. (And the reason why I am writing a lot of feedback mails to users lately.)

Workaround:
But at least there is a known "workaround" to overcome this new KitKat limitation (altough it seems that Google decided to remove this possibility with the update from 4.4.1 to 4.4.2). In Android 4.4 there exists a hidden settings screen ("AppOps") which allows you to grant or deny specific permissions to applications. You cannot open this settings screen directly from the system settings, but there are some apps in the Play Store which do nothing else, than just opening this (already existing but hidden) settings screen. For example, I used the app App Ops Starter. This simple app needs no permissions and does nothing else then opening Android's hidden "app ops" settings screen. Once opened, on the tab "Messaging", you see a list with all your installed apps which have sms-permissions. If you open the settings page for one specific app you can grant the permission "Write SMS/MMS" again and the system will behave again like it did before Android 4.4.



So, long story short:
It's been a long time with this problem in the Android world. Hopefully workarounds like my app should not be needed in future (when all SMS apps make use of the "date_sent" field in the SMS database).

Until then it would be possible to modify the SMS Sent Time app to overwrite the datestamp field in the database (instead appending the time to the message text). Any develper should feel free to take the source code and modify it accordingly.


[edit]
corrected spelling
[/edit]

 

[androcheck]

Hi!

Sorry for the delayed answer and thanks for your appreciation.

Just wanted to comment on your 3 points:

1) It is possible for 3rd party messaging apps like Handcent and Go SMS to implement the sent time either in message details or in the timestamp above the message.

Yes! Indeed! On every ROM which inherited the new SMS database layout from Google's original open source Android version 4.0.3 it should already today be possible for SMS apps to display both times: the sent time AND the received time!

So if your Android system database has this additional sent_time field, you just need a SMS app which shows it to you. So go ahead and send mails to your favorite SMS app developers and ask them about it (you may provide the link to the updated SMS database layout since Android 4.0 including the new "date_sent" field)

2) That hope vanishes with KK 4.4.2

No, this hope does not vanish! Only my app may not work any longer on 4.4.2 but if have already the sent_time in your normal SMS app you won't need my app anymore. (I'm probably the first developer being happy about decreasing user numbers. )

3) That hope could be restored if Android again revises its SMS database structure or the message handling process to capture and display the time the SMS was actually sent

No, that has nothing to do with it. As noted above, Google has already improved the database structure beginning with Android 4.0.3. Now just all phone manufacturers need to inherit this improvement (which I hope they already did - but I have no concrete infos here).

And we need all the SMS app developers (3rd party apps like Handcent and Go SMS but also the "default" messaging apps from Samsung, HTC, etc..) to display the sent_time (which hopefully is already there on every phone running Android 4.0.3 or higher).

So in general we are on the right way!


P.S.: When I use Google Hangout as SMS app I can see in the message details sent time AND received time. Can you try this just to verify if in principle it also works on your phone?

 

[Brianakin]

Principle confirmed. Hangouts does display both times in msg details. On to my favorite SMS app developer to request this option in a future update!

 

[Catbert2014]

SMS Sent time Issue, but which new phone?

Yes, many thanks Androcheck for the SMS Sent Time app. The issue is immensely irritating and I don't mind at all that the time sent is appended to the message itself. However, it's now time for a new phone (for a number of reasons) and I would welcome advice on whether it would be easier to:

1. Stick with Android and ensure the new phone has up to 4.4.1 only and not 4.4.2?
2. get a 4.4.2 and wait fo the developers to catch up and all start using the sent time field
3. get an iphone - no, have already ruled this out for various reasons
4. Consider a windows phone - do these have the same issue?

Advice anyone?

Thank you for reading my post

 

[4trackmind]

Lollipop

Hi. Any chance to have this one run on Lollipop? This is a big help on my Acer and Xperia phone previously running ICS. Please help. Thank you.

 

[treysis]

Lollipop & S4

I just want to confirm that if you turn on the SMS/MMS write permission for SMS Sent Time in App Ops, then this fine application is still working also in Lollipop 5.0.1 (at least on my S4 i9505 it is!). Unfortunately, App Ops need root! I had rooted mine before, but it's definitely a very fine reason for rooting the phone.

This is really one of the most useful apps for me on my android phone. I don't understand why this feature has always slipped through with the manufacturers. Just as an example: yesterday my boss sent me a text. Due to slow network and bad coverage, I received his text at 12:30 AM, asking to call him when I have network. Yeah...well...luckily he was not sleeping yet Had I known that it was sent hours before, I would have known that he had already reached one of my colleagues and a callback (at this time) was totally unnecessary.
So, I am very happy to find that it is working again! Of course, a timestamp field in the messaging app would be better, but for most people to confusing. So to append the sent time to the message seems like the 2nd best to me.

 

[eresac]

I see this is an old thread now but, since I've been redirected here from "SMS Sent Time" google play page, I'll use it anyway.
I've recently update to android 7.1.1 and the app doesn't seem to work anymore.