[Solved] Reverse Engineering a 32A Kernel [Updated 28th August]

[TigerTael]

New 32A Magic Kernel Patches Thread, Please upgrade to V3 patch!

Apparently HTC have released their kernel sources for the 32A magic @ http://developer.htc.com/

Thank you for your help everyone!

I have just reviewed the HTC kernel source and it looks like HTC actually pulled their thumbs out of their arse! Apart from the two different memory locations below, HTC have also added some logic to their kernel source which will probably be of great benefit to everyone.

There are two differences between my kernel patch and HTC's. They are:

Mine:
VMALLOC_END (PAGE_OFFSET + 0x19200000)

Theirs:
VMALLOC_END (PAGE_OFFSET + 0x20000000)

Mine:
MSM_EBI_SIZE 0x0CE00000

Theirs:
MSM_EBI_SMI32_256MB_SIZE 0x0C600000

You can't really blame me too much for these as they were impossible to reverse engineer. I could only come to a size based on differential calculation from the 32B source.

Also, those who know, will recall that I wasn't too sure about my EBI size.

::ROLOGUE:::
After many countless hours that I have spent bashing away at the keyboard, and many tears of blood, sweat and sorrow, we seem to finally have a booting 32A Kernel which seems to behave properly.

If anybody has any problems with the kernel and/or memory addresses, please can you forward them to me and I will look into it. OR you can post the details here, I'm sure we'll look at it. I'm sad to have to say this, but any memory location problems/kernel errors relating to memory issues are probably my fault. I did the best that I could. I am fairly confident that the memory addresses are correct, although there is that niggling in the back of my mind that there is something not right but judging by the response and dmesg and logcats, I have found no such evidence. If there are any problems with memory addresses and something ends up happening, burning down your house or whatever, I'm not responsible. Sure, I'll be more than happy to fix any mistakes (if there are any), but come on, cut me some slack.

In fact, blame HTC for not releasing the kernel patches (They are legally obliged to). I think that perhaps when they do end up releasing the kernel patches, only then will I find out how good my calculations were and even then, some things you would not be able to found out purely by calculations alone. Also, the memory layout for the 32A kernels is quite the opposite to the 32B kernels. Did they do this on purpose or was it a rush job?

Why HTC, WHY?
:::END:::

Result: GREAT SUCCESS!

What this means for the layman is that we will be able to finally get tethering support and any future kernel developments (ROM features) that are in 32B ROM's and we 32A users could not use them!

Developers, please note, after you apply this kernel patch, you will NOT be able to build 32B kernels from the same source. There have been quite a few forks.

Thank you rayman84 for the help! Special thanks to zinx @Freenode for your awesome diagnostic app to help discover ADSP/Camera!

To everyone with a 32A handset that is looking forward to this, feel free to try compile your own kernel but also remember that things are likely to change as more discoveries are made. If you want a stable 32A kernel with iptables and netfilter with all features working, you're going to have to wait a little longer. I'm working on this as much as I can and ALL help is appreciated from everyone! Fixing the kernel for the 32A platform will benefit everyone! If you feel you can help, please do! Any changes you make or suggestions will be taken seriously.

I have posted my own kernel patch for the latest 2.6.27 kernel. This is still experimental but everything seems to be working!

- - - - - This is for developers only and is very experimental! Backup your files, patch is (un)likely to change. - - - - -

2.6.27 Kernel Patch v2 - Link removed to prevent people downloading this for the time being. Please check the new thread for files.

- - - - - This is for developers only and is very experimental! Backup your files, patch is (un)likely to change. - - - - -

* Changelog:

V3 - COMING SOON
Changed Makefile.boot back to V1 - initrd_phys-y := 0x19A00000
Changed EBI_SIZE to the same size as the HTC released kernel source.
Changed VMALLOC to the same size as the HTC released kernel source.

V2
Changed Makefile.boot - changed to initrd_phys-y := 0x1A200000 (credit to rayman84)

---------------------------------------------------------------------------------------------------------------------
Credits:

* TigerTael - That's me! I spent hours picking away at the kernel, doing differential calculations, picking through dmesg's and logcat's. No donations necessary! This is for you guys!
* rayman84 - Thanks for your help! If you had not come along when you did, I might've spent countless hours without results if it weren't for you. (I forgot to edit my Makefile.boot)
* Radix999 - Thanks for your help man! I appreciated all the support.
* zinx @ Freenode - Thanks man! Your help made this possible!
* setenza01, kingchris, Wysie, thanks for all your dmesg/logcat/other herlp! We would've not gotten this far.
* gboddina -

gboddina's kernel with his kernel config and wlan module, iptables, etc. (Note: This includes a ramdisk, may not work properly on all ROMS. Have the ROM cooker integrate this kernel for you!). V4 apparently works just fine. V5 was giving me time issues.​

* #android @ Freenode
* biktor_gj - Thanks for chipping in and general help!
* Maxisma - Thanks bud!
* Thank you everyone else! We still need your help! (The Android community as well as any bug reports)


Things that are working:

* SDCard
* Memory - (All memory)
* Camera - (Special thanks to zinx @ Freenode!)
* GPU1 - (Pretty sure this is right)
* ADSP - (Pretty sure this is right)
* MDP - (Untested)
* LED's - working fine.
* GPS -working fine.
* Bluetooth - working fine.
* WLAN - Working 100%


Things that are not working yet:

* Unkown (Hopefully none)

 

[setenza01]

Sapphire 32B :

cat /proc/iomem
00700000-007fffff : msm_panel.0
10000000-164fffff : System RAM
10024000-10352fff : Kernel text
10354000-104293b7 : Kernel data
16d00000-16d1ffff : ram_console
a0200000-a0200fff : msm_serial_hs.0
a0400000-a0400fff : msm_sdcc.1
a0500000-a0500fff : msm_sdcc.2
a0800000-a0801000 : msm_hsusb
a9900000-a9900fff : msm_i2c.0
a9900000-a9900fff : msm_i2c
a9a00000-a9a00fff : msm_serial.0
a9a00000-a9a00fff : msm_serial
aa200000-aa2effff : mdp
aa600000-aa600fff : msm_mddi.0
e8100010-e8100017 : leds-cpld

 

[TigerTael]

@setenza01,

Thanks for that, it will help...

Could you possibly do 'dmesg' and pastebin it please? This is also important!

 

[setenza01]

Fore dmesg : http://pastebin.com/m14bad41a

 

[TigerTael]

@setenza01,

Thank you again. Hopefully with this I can backwards calculate the memory locations with the current sapphire source and then examine the dmesg from a working 32A kernel and calculate the differences.

 

[Wysie]

Good luck and thanks for the hard work! Are you working with biktor_gj?

 

[*child]

Godspeed

 

[TigerTael]

Good luck and thanks for the hard work! Are you working with biktor_gj?

At the moment, I'm not working with biktor_gj, but we have exchanged words in other threads and I found his other thread where he is attempting to get a 32A flashed with a 32B radio, up and running with all its memory.

My attempt is the opposite of this as I am trying to actually compile the kernel purely for a 32A with a radio that is made for 32A devices (I.E. stock radio). We're not exactly working together, but I will be keeping an eye on his attempts. Perhaps we can learn from each other and may end up cooperating. I'll be working on this tomorrow afternoon I think, as I have studies in the morning.

 

[Wysie]

Ah ok, thanks for the hard work ! Haha. Hopefully both of you or one of you find a way!

 

[shwan_3]

At the moment, I'm not working with biktor_gj, but we have exchanged words in other threads and I found his other thread where he is attempting to get a 32A flashed with a 32B radio, up and running with all its memory.

My attempt is the opposite of this as I am trying to actually compile the kernel purely for a 32A with a radio that is made for 32A devices (I.E. stock radio). We're not exactly working together, but I will be keeping an eye on his attempts. Perhaps we can learn from each other and may end up cooperating. I'll be working on this tomorrow afternoon I think, as I have studies in the morning.

Thnak you man! make it work and we (I) will donate you a drink for sure!

 

[TigerTael]

Thanks to setenza01 for the previous dmesg.

However, I need somebody else to do a 'dmesg' for me. setenza is using a newer kernel, 2.6.29, and I am currently trying to reverse engineer a 2.6.27 kernel.

I will try and use this dmesg, but one that is running a true 2.6.27 would be more beneficial.

 

[TigerTael]

I have been working on this for 6 hours or so, and I have made a very small bit of progress. The kernel is still not booting the phone, but now the kernel is rebooting the phone instead. Not very much progress, but I think I may be very close here.

There is something else I need though...

Can someone turn off their phone, hook it up to their computer, go to a console where they have adb installed and do this please:

adb logcat > logcat.txt

You will get a prompt telling you that it is waiting for device. Let the phone boot up all the way into android, where you can see the desktop and then ctrl + C in the command prompt window.

Can you then open logcat.txt and paste the entire thing to a pastebin. This is quite important, I feel that this is going to give me a few more crucial clues.

I need one from a 32B Magic/Sapphire/myTouch and a 32A Magic/Sapphire/Mytouch

If you know of anyone with either of these devices, please can you bend their ear to get them to do it. Also, please state which device you are running and possibly which ROM.

 

[Wysie]

Hiya, as requested:

http://pastebin.com/m735842f8

I have a 32A Magic, running Amon_RA's RAv1.1.0 . Hope it helps! Thanks for the hard work!

 

[kingchris]

http://pastebin.com/f119730bb

32b Magic running Cyanogen 3.9.10

hope it helps

 

[TigerTael]

@Wysie, @kingchris,

Thank you guys. These contributions will help. No more need for any logfiles guys, I have all that I'm going to be able to get out of these running phones/kernels.

Tomorrow, I will be correlating the data and trying to make sense of it all. It's a bit late and I'm getting moaned at for spending so much time with my phone.

But never fear, I will continue to work on this. Hopefully I don't meet with a dead end, but with the data so far I am quite hopeful. I hope to have good news for everyone soon.

 

[Wysie]

No problem . Hope to hear good news!

 

[hotweiss]

Oh wow, good luck man.

 

[TigerTael]

I've just stumbled onto something else that may help me.

I need a dmesg from an 32B HTC Magic/Sapphire/myTouch/etc running 2.6.27 kernel

If you have a phone running this kernel, PLEASE can you pastebin it or something. It is very helpful.

I have been working on this today but have not had much progress yet. It's taking a lot of my patience to work on this but I know that people need it and so do I. Hopefully I can come right with this.

 

[zambezy]

ok.. how do I export this to a file...

 

[TigerTael]

Okay, another 7 hours later, I managed to figure out some more memory addresses, but I am still having trouble with this.

Looking through a 32A dmesg, there is something else which is present which is not present in a 32B dmesg. Here is one of the main difference which I believe is responsible for not letting me boot:

<4>[    0.595220] +setup_mdp_sync_config()
<4>[    0.595251] mdp_base=cc900000, phys_to_virt=51000000
<4>[    0.595281] Enabling MDP_CLK; GLBL_CLK_ENA=221bfe2f
<4>[    0.595312] set MDP_SYNC_CONFIG_0 from ffcfffff to 7af01192
<4>[    0.595312] set MDP_SYNC_THRESH_0 from 40004000 to b0b

There is some other function that HTC made which is not in the source of the kernel. Unfortunately, 0xcc900000 doesn't really make much sense to me unless it's some other graphics buffer. Perhaps this is just some more debug info? I'm not sure. I have gathered all the memory locations I have and will post them now.

Things that I am very sure about the 32A Kernel (That is working):

MSM_PMEM_GPU0 		Start	0x0
MSM_PMEM_GPU0 		End	0x700000

MSM_FB_BASE		Start	0x700000
MSM_FB_BASE		End	0x079B00

MSM_RAM_CONSOLE_BASE 	Start	0x7A0000
MSM_RAM_CONSOLE_BASE 	End	0x7C0000

MSM_SMI_BASE		Start	0x0
MSM_SMI_BASE		End	0x800000

SMI32_MSM_LINUX_BASE	Start	0x19200000
SMI32_MSM_LINUX_BASE	End	0x25800000

MSM_PMEM_GPU1_BASE	Start	0x25800000
MSM_PMEM_GPU1_BASE	End	0x26000000

EBI_BASE		Start	? 
(AFAIK, should be where Linux base starts, but possibly needs to envelop ram_console)
EBI_BASE		End	? 
(AFAIK, should be where Linux base ends or perhaps slightly beyond)

Things that I am unsure about:

1) I am not sure if there are 1 or 2 memory banks, so I have had to test them both but there are so many variables. Knowing for sure how many would definitely help.
2) Additional code besides the memory addresses and the memory fixup seems to be missing. Function "+setup_mdp_sync_config()" is nowhere to be found and I think this is the mean reason why I am not getting anywhere.

I will continue to work on this, but without these functions I'm not sure if I'm going to get this working without it. I am posting this information here just so that people who might be able to suggest something to try can give an educated guess.

Okay, I gotta get to bed. It's getting late, work tomorrow.

Just FYI, when you compile the kernel, it subtracts 1 from the end point so as not to have memory overlapping.