[KERNEL] Moxie N7 / M12

[mixtapes08]

Yep, Nexus 4. I think I saw you over in that forum too IIRC. Then I got sick of it all, and rocked a dumb flip phone for a while. I thought I was in the matrix. Then I got a HTC One X plus, but it was too heavy, and I really didn't think the beats audio was that impressive. In fact I thought it sounded bad. Incidentally, I think the audio from the jack in the op3 sounds very good!

I later had a go with a Nokia 830, which was an excellent device, but as everyone says, Windows Mobile is a poor ecosystem. The OS itself was a bit ropey too.

I also had a Moto G 2nd gen at some point. Another solid device, but a bit too discrete for my taste! I then had a Sony X3 compact. A very nice device! But I broke the screen

I'm sure I'm forgetting some. But I felt the op3 was a total bargain at the price. I couldn't resist. It's a bit short on battery, but that's my only gripe.

Yeah z3 compact was a solid device. Solid battery afaik, my friend got one then decided to switch to nexus 5x that had mediocre battery. Lol. Anyway op3 is the best choice today. I think it's the fastest phone up to date. I'm very satisfied with it.

Sent from my ONEPLUS A3000

 

[bedalus]

Okay, as promised, I'll go into some detail about what I've achieved in this update. First I want to talk about this file: drivers/gpu/msm/adreno_cp_parser.c

See Github commit: 9464eba
I just want to go over the methodology first. Since the kernel selects CC_OPTIMIZE_FOR_SIZE in the def_config, warnings about potentially uninitialised variables are disabled in the Makefile:

ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
KBUILD_CFLAGS	+= -Os $(call cc-disable-warning,maybe-uninitialized,)

This has been standard for a while I think, since it produces a lot of false positives. I think I had to sift through around 70 files, picking up a few hundred uninitialised variables. However, it seemed to me that there were a few functions that appeared to have genuine issues. The worst offender appears to be this file adreno_cp_parser.c

for (i = 0; i < ib_obj_list->num_objs; i++)

There are four instances of this loop in this file, each following a similar pattern of looping through items in an array, however, the original code appears to be lacking a scope (in C a scope is a section of code that is contained within curly braces {}) for this one loop. This has the effect of failing to perform any of the checks in the following if statement:

if ((SNAPSHOT_GPU_OBJECT_IB == ib_obj->snapshot_obj_type) &&
  			(gpuaddr >= ib_obj->gpuaddr) &&
  			(gpuaddr + dwords * sizeof(unsigned int) <=
  			ib_obj->gpuaddr + ib_obj->size))
  			return 0;

...for all objects excepting for the final one in the array.

The rest of the code changes (the other files in the same commit) were more trivial, just a bit of code shuffling really. These minor changes merely make it absolutely certain that the variables must have been initialised before being used. However, it's perfectly reasonable to assume that they could never execute without being uninitialised because of the logical flow of the functions, but there's nothing wrong with making certain.

 

[bedalus]

Newer testing version: here

For Testing Only!: http://d-h.st/5JE2

I've merged in all applicable commits from both the linux kernel mainline 3.18.y LTS updates branch, and also (more experimentally) all the applicable commits from android kernel common for 3.18

There's a lot going on. The kernel appears to work, but I need to have a good look through the commits to make sure everything I've applied was sensible, and also give it a good testing to ensure nothing is broken. Watch out for kernel Oopses, these will manifest as a sudden reboot.

It's interesting to note that back five years ago we couldn't merge in LTS patches so confidently as android wasn't accepted into the mainline kernel. Fortunately it is now, and has been for a good length of time, so patches are quite likely to apply cleanly without breaking anything. However, since we pull from kernel source modified by OnePlus, we inevitably encounter some code that has been altered already.

My approach to code conflict is to defer to the OnePlus changes. I cherry-pick my way through the individual commits using a script and automatically abandon any patches that conflict with the OnePlus source.

It's also interesting to note that a great deal of patches don't have any impact on code that is actually built for our device, since the full linux kernel source supports many architectures and devices that have nothing to do with our device. However I still bring these commits in since subsequent patches that will affect our code may rely on earlier patches that had nothing to do with our device's code!

Here's some useful commands for helping to understand the changes brought in from other sources in large batches:

find . -name '*.o'
git diff --stat c9d75816a8b1118452926b3455f56418a703d1d1..HEAD

First you list all of the .o files, which are the objects build by the compiler from the actual .c code files that we require for a kernel that works on our device. The second line produces an output like this:

arch/arc/Makefile                                        |   1 -
 arch/arc/include/asm/entry.h                             |  17 ++--
 arch/arc/include/asm/irqflags.h                          |   2 +-
 arch/arc/include/uapi/asm/elf.h                          |  11 +-
 arch/arc/kernel/arcksyms.c                               |   2 +
 arch/arc/kernel/asm-offsets.c                            |   2 +
 arch/arc/kernel/process.c                                |   2 +-
 arch/arc/kernel/setup.c                                  |   6 +-
 arch/arc/kernel/stacktrace.c                             |   2 +-
 arch/arc/mm/cache_arc700.c                               |   9 ++
 arch/arm/Makefile                                        |   8 ++
 arch/arm/boot/dts/armada-375.dtsi                        |   2 +-
 arch/arm/boot/dts/dra7.dtsi                              |  71 +++++++++++++
 arch/arm/boot/dts/imx25-pdk.dts                          |   5 +-
...etc...

...that shows us which files have been changed by the patches from a certain point (c9d7581 is after I'd applied all of the mainline patches, but before I'd started patching from android-kernel-common), up to 'HEAD' which is the latest commit. You can then use a tool like MS Excel to compare the two lists and see what's changed in just the files in our device's kernel. It's the same as the overlapping bit in a Venn diagram!

So, using this method I've worked out the details of this epic code cherry-pick. From the linux mainline we have 30794 lines of code changed, of which:

code		removed	added
OP3		11.8%	14.2%
N.A		28.0%	46.0%

From the android-kernel-common repo we have 28216 lines of code changed, of which:

code		removed	added
OP3		12.0%	21.0%
N.A		5.6%	61.5%

It's also interesting to observe that although the mainline has patched a higher volume of code, the amount that is applicable to the device we're building for is about 15% less than kernel-common has provided (8000 lines vs 9300 lines). It's usual for code to grow slightly over time too, as more comprehensive safety checks and features are brought in. Often too, more efficient algorithms are longer in code length, so sometimes performance improvements result in longer code. Another observation is that the mainline code additions that apply to our device have only caused a total growth of about 740 lines vs kernel-common, which has introduced ~2500 new lines of code. This is as you would expect, since the mainline caters to everyone and has a LTS remit, i.e. to provide stability, whereas android is much more targeted and is always adding new features that require kernel support.

So what were the biggest affected areas from this big patch from kernel-common?

directory		removals	additions
 drivers/md		997		1309
 fs/ext4		440.75		1163.25
 mm			437.25		1064.75
 init			13.5		450.5
 security/selinux	161		314
 drivers/cpufreq	0		54

In my next post I'll provide some additional detail about what changes have been provided in the new code added to these areas of the kernel.

 

[kentexcitebot]

Wow, I'm actually learning a lot by reading this thread! I think developers on XDA should write these detailed posts, definitely interesting @bedalus

 

[DontPushButtons]

Well now, I was putzing around here checking out development as motivation to buy an OP3, and I find @bedalus has come out of hiding again. How you been man, haven't seen you since moob on mako.

Sent from my Nexus 4 using Tapatalk

 

[bedalus]

Well now, I was putzing around here checking out development as motivation to buy an OP3, and I find @bedalus has come out of hiding again. How you been man, haven't seen you since moob on mako.

Well, as you can see I've re-joined XDA... I still enjoy kernel development, and I like to share my work (and get help testing!)

I think since mako I've radially altered career path, from maths teacher to reporting analyst. I'm making fair money, but I enjoy the work so it feels like I'm robbing them! The thing about teaching maths was (sigh) you never actually get to do any! But my current job has problem solving at its core. And I'm basically burning the candle at both ends because after work I'm coming home and slamming straight into git and C to build this project.

I just need to go off on a tangent about numbers, since I've been listening to Feist on a heavy rotation, and I just noticed my thanks meter has hit 303: Feist, flight #303 (youtube)

...just a weird coincidence!

What about you? Seen any amazing development? Tempted by the OP3?

 

[AlkaliV2]

Okay, as promised, I'll go into some detail about what I've achieved in this update. First I want to talk about this file: drivers/gpu/msm/adreno_cp_parser.c

See Github commit: 9464eba
I just want to go over the methodology first. Since the kernel selects CC_OPTIMIZE_FOR_SIZE in the def_config, warnings about potentially uninitialised variables are disabled in the Makefile:

ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
KBUILD_CFLAGS	+= -Os $(call cc-disable-warning,maybe-uninitialized,)

This has been standard for a while I think, since it produces a lot of false positives. I think I had to sift through around 70 files, picking up a few hundred uninitialised variables. However, it seemed to me that there were a few functions that appeared to have genuine issues. The worst offender appears to be this file adreno_cp_parser.c

for (i = 0; i < ib_obj_list->num_objs; i++)

There are four instances of this loop in this file, each following a similar pattern of looping through items in an array, however, the original code appears to be lacking a scope (in C a scope is a section of code that is contained within curly braces {}) for this one loop. This has the effect of failing to perform any of the checks in the following if statement:

if ((SNAPSHOT_GPU_OBJECT_IB == ib_obj->snapshot_obj_type) &&
  			(gpuaddr >= ib_obj->gpuaddr) &&
  			(gpuaddr + dwords * sizeof(unsigned int) <=
  			ib_obj->gpuaddr + ib_obj->size))
  			return 0;

...for all objects excepting for the final one in the array.

The rest of the code changes (the other files in the same commit) were more trivial, just a bit of code shuffling really. These minor changes merely make it absolutely certain that the variables must have been initialised before being used. However, it's perfectly reasonable to assume that they could never execute without being uninitialised because of the logical flow of the functions, but there's nothing wrong with making certain.

I've been running your test kernel for about 13 hours uptime and there have been no random reboots or instability. I checked the kernel logs for errors and other than some Socks callbacks (probably due to the school wifi I'm using) the only error I was seeing at kernel level was the synaptics call-out in the screenshot below. Don't know if it has any effect, but overall the kernel is running very well. I haven't been able to play with it much because of classes but this weekend I'll try using it the normal amount and see if I have any issues. Thanks for sharing the test build, it is doing very well so far! :good:

 

[bedalus]

I've been running your test kernel for about 13 hours uptime and there have been no random reboots or instability. I checked the kernel logs for errors and other than some Socks callbacks (probably due to the school wifi I'm using) the only error I was seeing at kernel level was the synaptics call-out in the screenshot below. Don't know if it has any effect, but overall the kernel is running very well. I haven't been able to play with it much because of classes but this weekend I'll try using it the normal amount and see if I have any issues. Thanks for sharing the test build, it is doing very well so far! :good:

What app is that please? I like how the colour scheme highlights the log info. Looking at the messages, they are just ordinary debug information rather than errors. It's normal for the kernel to log benign messages. Often kernel hackers disable debug logging as they see it as an unacceptable overhead (even though it is really tiny!). If it looks like these messages are appearing too much. I'll remove the code. Here's the function in question. You can see the TPD_DEBUG lines are just logging suspends/resumes/screen blanks. Nothing to worry about!

I'm very grateful to have another tester actually! Thanks for posting your insights.

If you expect a certain amount of battery consumption, would you be able to recognise if it was less or more than usual?

 

[nevermorenorless]

What app is that please? I like how the colour scheme highlights the log info. Looking at the messages, they are just ordinary debug information rather than errors. It's normal for the kernel to log benign messages. Often kernel hackers disable debug logging as they see it as an unacceptable overhead (even though it is really tiny!). If it looks like these messages are appearing too much. I'll remove the code. Here's the function in question. You can see the TPD_DEBUG lines are just logging suspends/resumes/screen blanks. Nothing to worry about!

I'm very grateful to have another tester actually! Thanks for posting your insights.

If you expect a certain amount of battery consumption, would you be able to recognise if it was less or more than usual?

It looks to be 3c toolbox

 

[AlkaliV2]

What app is that please? I like how the colour scheme highlights the log info. Looking at the messages, they are just ordinary debug information rather than errors. It's normal for the kernel to log benign messages. Often kernel hackers disable debug logging as they see it as an unacceptable overhead (even though it is really tiny!). If it looks like these messages are appearing too much. I'll remove the code. Here's the function in question. You can see the TPD_DEBUG lines are just logging suspends/resumes/screen blanks. Nothing to worry about!

I'm very grateful to have another tester actually! Thanks for posting your insights.

If you expect a certain amount of battery consumption, would you be able to recognise if it was less or more than usual?

He's correct, it is 3C Toolbox (I have pro) and it does a really good job with logs and general tweaks for our OS. I used it exclusively for the longest time.

I'm glad to do testing, I'm learning Java programming at the moment in my classes and you posting your code snippets and changes is awesome because I get to see the real-world application of the logic. I am able to tell if there are battery changes but more in idle lossage based on how I treat my device. If I see any drastic changes I ensure it isn't my apps causing the issue because earlier I was experiencing lag but it was LastPass causing the issue.

Anyway, I'll keep testing and let you know if anything changes. So far it's been up for the entire day so stability is not an issue. Again, thank you for posting your knowledge and development steps. I get a lot out of it.

 

[XeloOw]

Yeah this is very cool that the kernel dev shows to us what he changed in the code .
I really appreciate that !

 

[bedalus]

...In my next post I'll provide some additional detail about what changes have been provided in the new code added to these areas of the kernel.

I've set up an online sheet that shows the stats from the two big patch sets in more detail here

I've had a look through all the commits to see what the most interesting changes are. The majority are simple patches to optimise code or provide extra safety checks. These four patches are the most interesting because they are a bit more radical and pertinent to our device due to its limited battery capacity:
8b9a0bb

6c2f52a

aa5c87c

63fe3af

These patches improve the accuracy of the CPU load calculations, which is a crucial factor in determining the need to scale the CPU frequency up or down. CPU frequency is one of the biggest factors in power consumption.

Back in the Nexus S days I did a lot of work measuring battery drain and if you're interested in what is actually effective in saving power you can still get a lot of useful info from reading the first four posts in the old thread here. In short, the biggest drain after the screen is the CPU (and if you didn't know, power can be saved in OLED screens by using black and red themes, the more black the better).

Incidentally, under-volting the CPU essentially does nothing, except make it likely that your device will crash and reboot, ironically causing an enormous drain on the battery. But the CPU can save a significant amount of power by keeping the frequency low. This doesn't mean you deliberately hamstring your device, but there are other factors at work, like the bus speed, and what the GPU can handle.

One of the most effective mods I developed on the Nexus 4 was to allow higher bus speeds at lower CPU frequencies. This meant the GPU could work at its full capacity while the CPU was using less power. This will be the next stage of development if the current testing version proves itself to be stable.

Edit: For my own records, https://source.codeaurora.org/quic/la/platform/frameworks/native/refs/tags?h=LA.AF.1.1.1 recently pushed LA.HB.1.3.2-28400-8x96.0 (wifi)

 

[Stroky]

Wow this sounds pretty awesome! Looking forward for you results!
Really interesting.

 

[mady51]

Damn getting to know many things from these posts which is awesome. Loving it bro. Keep going and rock on

 

[Antsu]

Test kernel flashed on stock 3.2.7... It works . And to be precise, I've encountered zero problems yet. Thanks. I'll be using this until your next test release and report if something unusual happens. I'm not a pro tester or even novice, but I'll try my best to produce some logs, if there's found any bugs.

 

[bedalus]

Another one to test please: http://d-h.st/h25y

 

[AlkaliV2]

Another one to test please: http://d-h.st/h25y

Just got it installed, wiped cache/dalvik, and I'm giving it a full charge before putting it through its paces. I'll let you know if the logs tell of any issues after I've had some time to play with it. Never had any issues with the last one over the past two days.

I read your post about allowing the GPU to work at full speed by raising the bus speeds while keeping the CPU clock low, but I wanted to ask how much of the rendering is handled by the GPU. When battery saving mode kicks in, it turns off haptic feedback, animations/transitions, and brightness is lowered (I assume CPU frequencies are governed as well). I suppose my question is: Does the GPU and CPU handle the UI draws and smoothness of the interface or is it biased one way or another? I know touch boost exists which boosts the CPU quite a bit, but I assumed that was for passing commands to the OS not necessarily for scrolling purposes.

 

[AlkaliV2]

There's been a lot more feedback in this build from the kernel logs. I filtered it to show only warnings and errors so that is what is attached.

Warning and errors aside the performance of this kernel for me was insanely good. With my transitions and animations set to .5x my phone is extremely smooth on this test kernel release. My first reboot it was like a brand new install it was so fast. Idle loss was VERY low (less than 2%/hr) but screen on loss was higher on this release. I suspect this has to do with Allo because it was at the top of my battery stats using a little more than 3x the mAh of the screen.

Overall performance is astounding with me applying zero tweaks to the kernel. Hopefully the logs only lead to improvements.

 

[Antsu]

There's some error logs. Didn't notice anything unusual while using the phone and while those errors happened. No freezings, lags and reboots. Screen on and off drain is about same level than with previous test version. Overall performance very good. I've put windows- and transition animations off and animator to 0.5. Interesting how quick the phone can be.

Edit. More logs. Nothing happened though.

 

[Antsu]

Screen off battery consumption has been just decreasing, now it's 0.8%/h, with the latest test version. Screen on consumption is about 10%/h. Adaptive brightness is on, at lowest level.