[BUG FIX] Phantom keypress and screen shot

I've been working on fixing this issue for awhile. Here's the deal:

The problem.
The four keys at the bottom of the phone are monitored by a melfas touchkey chip (http://www.melfas.com/english/touch/sensor.asp) that connects to the main processor via an I2C bus (http://en.wikipedia.org/wiki/i2c). The melfas chip generates an interrupt whenever one of the keys is touched or released. The processor then reads the key value from this chip over the i2c bus. The problem is that the touchkey chip is located right next to the 3G antenna. When the phone is accessing the 3G network the RF energy gets transferred to the interrupt and i2c clock and data lines causing false interrupts to occur. The processor responds to the interrupt by reading the key value from the cypress chip. The symptoms occur more frequently in low signal areas because the phone outputs a higher RF level in those situations which causes more RF interference on the interrupt line.

Most of the time when a false interrupt has occurred the touchkey chip will return a value of zero for the key and the driver will recognize this as a bad key press and ignore it. Sometimes the RF interference on the i2c clock and/or data line causes a valid value to be returned and the driver reports a key press value to the application. In the case where the driver reports a ‘back’ key down, the software sees this as holding the back key down so when you press the power button you get a screen shot. The easiest way to cure this is to always press and release the back key before pushing the power button. This causes the software to see both a key down and key up event which cancels the screenshot mode.

This RFI induced touchkey interrupt happens hundreds of times per second when the phone is using 3G. It produces lots of different symptoms including applications that always seem to shut down. A wide variety of problems can be attributed to this failure. In addition, the processor spends a lot of time servicing these bogus interrupts, which take cpu time away from the other applications. This can make the phone appear to be slow or even freeze up for short periods of time. There’s a good chance that most people have experience this to some degree without realizing the root cause.

Solution one. Fix the driver.

Since this is a true hardware failure, a software solution is going to be less than perfect. After dozens of experiments rewriting the interrupt service routines in the driver I’ve settled on a combination of fixes. The first is to re-test the interrupt input line several times. In normal operation when you touch or release a button, the touchkey chip drives the interrupt line low and keeps it low until the driver reads data over the i2c interface. Since the RF interference is a sine wave and is being sampled it causes the interrupt line to go high and low at a fast rate. Sampling the line multiple times in software increases the chance of finding it in the high state. This is done both in the interrupt handler and then again in the interrupt thread. About 90% of the false interrupts are filtered out by testing the line in the handler. If the interrupt handler doesn’t find the line high after 10 samples, it masks the interrupt so that another falling edge doesn’t produce another interrupt. In testing I’ve noticed that the interrupt handler would run multiple times before the interrupt thread was even called. Once in a while, so many interrupts would get stacked up that the phone would just reboot. It was probably a stack or buffer overflow that wasn’t being handled. Remember, this interrupt would happen many hundreds of times a second. About 90% of the remaining false interrupts are filtered out by sampling this line in the thread. That leaves about 1% of the interrupts that need to be further tested. The second test is to read the data from the chip and discard anything that isn’t a valid key press value. This is easily done with a case statement. Finally, since occasionally a bogus valid value will get through, I set up a timer so that any key down event that doesn’t have a corresponding key up event within 3 seconds is canceled by calling the all_keys_up routine.

This combination all but eliminates the symptoms produced by this failure. The only draw back is that the processor still spends a considerable amount of time servicing the false interrupts. And rarely a phantom keypress does get through. In all, it’s a fairly good piece of duct tape and JB Weld.

During my experiments I used a copy of the kgb kernel. My version with the modified driver is in github at https://github.com/dmriley/kgb. If you want to try this yourself, be sure to use the ‘dev’ branch.


Solution two. Fix the hardware.

There are three signals that connect from the melfas touchkey chip to the processor. They are the two i2c lines: sdc which is the clock and sda which is the data. The third line is the interrupt. In troubleshooting this problem, I took my phone apart and put oscilloscope probes on the three lines. This allowed me to see the real cause of the problem. Since the interference is RFI (or EMI) the only real way to fix the problem is to either remove the RF or make the impedance of the signals much lower. Removing the RF is easy if you don’t need to use 3G. When the phone is using wifi (or no network connectivity at all) the problem does not exist. Also, when you are very close to a cell tower, the phone transmits at a much lower level. This lower level greatly reduces the RFI. Lowering the impedance is a little harder. I2C uses active pull down and passive pull up for the logic levels for both sda and sdc. This means that the impendence is mostly governed by the pull up resistor. This resistor value is typically upwards of 1kohm and probably as high as 3kohms (I didn’t measure it in this phone). Since the impedance only needs to be lowered for the 3G frequencies of around 800MHz, a capacitor can be added from the signal source to signal ground. At 800MHZ a 100 pf cap is about 2 ohms (1/ 2*pi*f*c). That’s a couple of orders of magnitude lower than the pull up resistor alone, and much too low for the RF signal to induce any significant voltage on the line. This value is also low enough not to interfere with the signal rise and fall times for the interrupt line. In the case of the interrupt line, the melfas chip drives the signal low and keeps it low until the interrupt is serviced. Discharging a 100pf cap with a 2mA driver takes only microseconds. This much delay is not noticeable when touching the key and is much less than the amount of time that the processor takes to service the interrupt.

Adding the cap to the interrupt line eliminates false interrupts. A chance does exist that a valid key event during 3G access could cause an incorrect key value to be returned due to RFI on the clock and data lines. The i2c protocol is designed to compensate for capacitive loading on the lines. Although it would cause the clock period to be stretched out significantly it would still only take milliseconds to read the key data from the chip. The difference would be imperceptible. To date I have only added the cap to the interrupt line and have yet to experience an invalid key press.

I’ll post pictures of cap mod.

Summary.

Most people will be satisfied using the software fix. I think that a couple of the kernel devs are incorporating some or most of the driver mods outlined in this document. Both comradesven (kgb dev) and ssewk2x aka Efpophis (glitch dev) were involved in the test and debug process. Much appreciation is given to both of them for the help that they gave me and for allowing me to use and hack up their code on github. Efpophis saved me hours of searching through code. Without their help, I’d still be unable to build a kernel.


UPDATE:30 Mar 2012
The phone had been working fine since the mod. I hadn't seen a screen capture or any of the other symptoms. Then, a couple of nights ago, while I running maps on 3G (a data intensive app) the touchkey backlights started flashing rapidly like the phone was having a little seizure. And then it happened, the voice search popped up. A couple of debug kernels later I've come to the conclusion (and I'm never wrong) that the clock line (SCL) going to the melfas chip was being toggled by the same RF interference that was causing the false interrupts. A random clock along with random data was causing the chip to turn the backlights on and off as well as generate a false interrupt. I was able to reliably duplicate the problem in a couple of really low signal level areas (not hard to find when you live out in the boonies).

I tore the phone apart (again) today and added a 100pf cap to the scl line right next to the chip. I also added another cap in parallel with the 100pf on the interrupt line. I spent about 1/2 hour tonight running 3G data apps in the same location where the problem first appeared. So far, no problems and none of the debug messages have shown up on dmesg.

If anyone wants pics of the added cap I'll open it back up, no problem, otherwise if you look at this photo you can see which pin is scl (although I incorrectly labeled it SDC in the photo). http://xdaforums.com/attachment.php?attachmentid=953824&d=1332117055

If anyone tries these mods I'd be real interested in your results.

the cap. yeah, that's a normal size pen to show scale
View attachment 951812

on the board
View attachment 951821

with notes
View attachment 951820

the antenna problem
View attachment 951822

close up showing touckey circuit. micro sd card for scale
View attachment 951834

my finger
View attachment 951836

back off
View attachment 951838

another view
View attachment 951837


BTW, I took these pictures with my son's fascinate

sendan said:

Uuuhm...You're an awesome human being. Holy crap. -_-

That's some amazing work, thanks!

Click to expand...

Click to collapse

wasn't just me. had help from other members here. I didn't even know where to start looking when I first started. It's so cool that people are willing to do the level of work that the devs here do without expecting anything back.

Samsung SPH-D710

This thread provided the information I needed to get Sprint (and Samsung) on board with a solution for the Galaxy S II 'Epic 4G Touch' model number SPH-D710.

Using this thread as a starting point, I that the problem with the Sprint S2 phones is also caused by RF interference. Here is the reply:

When the radio turns up power to maintain connection with the towers it can cause Electronic Noise to leak over the connection between the capacitive buttons at the bottom of the handset and the main board, making the phone think that there was a key press. This is due to a piece of shielding tape on the device not being up to snuff. The tape can't be replaced on its own in store, the entire screen assembly needs to be replaced.

This issue took quite a bit of time to track down and determine a root cause by Sprint/Samsung because of the large number of devices in the field with varying parts since launch and the multitude of potential causes. It needed to be determined how widespread the issue was, and the different types of hardware affected by this versus other issues with potentially similar symptoms.

The MENU button being "pressed" on its own is considered a known issue and can be fixed in a Service and Repair store by replacing the screen assembly (The replacement parts from Samsung are a combined LCD/TSP assembly from the factory so they are both replaced at once). It does appear more apparent in low signal/roaming environments due to the nature of the issue. If a store does not have parts available or is unable to fix it, they can exchange the device for you. Exchanges will be for the same model refurbished device, just as if you went through Samsung to haev it repaired/replaced. The refurbished devices should have the fixed part in place already from the warehouse. If you do not have TEP/ESRP from Sprint the $50 repair fee is waived for the repair/exchange for this issue. I do not know a specific date that this was posted in the internal device solutions portal, I just checked it today specifically for this issue.

Click to expand...

Click to collapse

Thanks to the OP of this thread and all those who contributed to its awesomeness!

neh4pres said:

Thanks so much for all the work, and the detail in your post. It is amazing the work everybody does here and the knowledge you pass on to us.
I do have a few questions


Would you mind sharing what kind off iron you used? is that the most bottom line on the board you soldered to? If so, did you have to scratch it or something first? Is it the farthest left line on the chip that was used? Do they make caps that size with leads coming of the 2 sides, and if so would that be a easier mod? Is there a positive and negative side to that capacitor?

I'm really thinking about doing this, if i decide to would you mind sending me 5 of your extra caps for a $10 donation?

Sent from my SCH-I500 using xda premium

Click to expand...

Click to collapse

I did the mod at my workplace under a microscope. I used a metcal (http://www.okinternational.com/product_soldering/mx500) soldering iron but you could use just about any low wattage iron with a really fine tip.

There's four pins on each side of the melfas chip. One end of the cap is soldered right to the interrupt pin which is the closest to the corner. the other end is connected to the ground side of C2 via a solder bridge.

View attachment 953824

I doubt that they make caps that small with leads on them. You could look. It's not hard to make the solder bridge. Remember the scale that were talking about here. That cap is 0.06 inches long by 0.03 inches wide. I wouldn't try to scratch the solder resist from the board because it's a flex circuit on top. Also, the cap is not polarized.

I bought a hundred of these caps for less than $6 including shipping. I'd feel terrible charging someone $10 for five. If you pm me your address I'll stick a couple in an envelope and send them. If you want to give away ten bucks, donate it to a charity like destiny rescue or UMCOR (http://new.gbgm-umc.org/umcor/about/financialinformation/).

Disclaimer:I've been working with parts this size for years and am pretty good at soldering. You risk dorking up your phone if you don't do this correctly. Only attempt if you are skilled at soldering. All information is presented "as is" and without warranty for fitness or use. Your mileage may vary. Void where prohibited, taxed or licensed.