FORUMS
Remove All Ads from XDA

[KERNEL][MODULE] CoDel kernel modules for Android (fighting bufferbloat)

131 posts
Thanks Meter: 208
 
By lioux, Senior Member on 24th February 2013, 02:06 PM
Post Reply Email Thread
Hi,

CoDel has been making its rounds around the Internet. It would be nice if we could have it on our Galaxy Nexus. In case you have no idea what I'm talking about:

Codel
Codel on BufferBloat.net

A must read, a non-technical explanation on BufferBloat, why we should care, why/how we should fight it and CoDel:

Why should we fight bufferbloat?

I apologize but I can't post URL links yet. The forum won't allow it.

This little "guide" teaches how to build CoDel based SCH_NET modules for Android. I'll probably should have posted on "Galaxy Nexus Android Development" but I don't have permission yet.

I've successfully built it for Android but I'm currently having trouble loading the modules on Android. I'm getting "exec format errors" when I insmod. I am pretty sure it's a version mismatch problem between my kernel source and my phone installed kernel. Anyone knowledgeable on generating kernel images willing to build this one? It should work if you build this against your kernel and generate a complete kernel image.

I used a kubuntu system for building the modules, yet you should be okay with any linux flavor.

You are required to know how to build a Android kernel module. It's assumed you know your way around a Linux system. This is not a step-by-step guide. It requires changes depending on your system installation.

Get the appropriate Android build toolchain and kernel sources for your Android.

I'm using the Linaro cross platform toolchain which you can download.

CoDel is obtained from the mcgrof compat module done by Luis R. Rodriguez. I only tweaked the Makefile to build it for Synology.

Download the mcgrof compat module.

Extract it somewhere. Rename the extracted directory to "mcgrof-compat" (without the quotes).

Apply the following patch for a kernel 3.0.+. I'm building for my Galaxy Nexus.

Code:
diff -ruN ../mcgrof-compat-5461c02.orig/compat/Makefile ./compat/Makefile
--- ../mcgrof-compat-5461c02.orig/compat/Makefile       2013-01-21 22:01:41.000000000 -0200
+++ ./compat/Makefile   2013-02-24 10:06:23.268068197 -0300
@@ -2,6 +2,10 @@
 obj-m += compat.o
 #compat-objs :=
 
+CONFIG_COMPAT_NET_SCH_FQ_CODEL=m
+CONFIG_COMPAT_NET_SCH_CODEL=m
+CONFIG_COMPAT_KERNEL_3_3=y
+
 obj-$(CONFIG_COMPAT_FIRMWARE_CLASS) += compat_firmware_class.o
 obj-$(CONFIG_COMPAT_NET_SCH_CODEL) += sch_codel.o
 
diff -ruN ../mcgrof-compat-5461c02.orig/Makefile ./Makefile
--- ../mcgrof-compat-5461c02.orig/Makefile      2013-01-21 22:01:41.000000000 -0200
+++ ./Makefile  2013-02-24 10:06:58.480242839 -0300
@@ -38,7 +38,7 @@
 # the same.
 NOSTDINC_FLAGS := -I$(M)/include/ \
        -include $(M)/include/linux/compat-2.6.h \
-       $(CFLAGS) \
+       $(CFLAGS) -fno-pic \
        -DCOMPAT_BASE="\"$(COMPAT_BASE)\"" \
        -DCOMPAT_BASE_TREE="\"$(COMPAT_BASE_TREE)\"" \
        -DCOMPAT_BASE_TREE_VERSION="\"$(COMPAT_BASE_TREE_VERSION)\"" \
Set CROSS_COMPILE to your toolchain and KLIB_BUILD to your kernel sources.

Code:
# cd mcgrof-compat
# ./scripts/gen-compat-config.sh > mcgrof-compat/compat/.config
# ./scripts/gen-compat-autoconf.sh mcgrof-compat/compat/.config > ./include/linux/compat_autoconf.h
# make ARCH=arm  \
	CROSS_COMPILE="/v/0/ccache-bin/arm-linux-androideabi-" \
	KLIB_BUILD=/v/0/android-git/linux-linaro-3.0/
# ls mcgrof-compat/compat/*ko
Now, you have the following kernel modules:

mcgrof-compat/compat/compat.ko
mcgrof-compat/compat/sch_codel.ko
mcgrof-compat/compat/sch_fq_codel.ko

Decide which CoDel flavor you need: codel or fq_codel. Copy the files to your Android.

For example, to load fq_codel which is a fair queueing implementation:

Code:
# insmod compat.ko
# insmod sch_fq_codel.ko
# tc qdisc add dev rmmnet0 root fq_codel
# tc qdisc add dev wlan0 root fq_codel
I hope this is helpful to you. You now have both codel and fq_codel modules and you can load them.

Codel is not a silver bullet: it needs at least basic configuration (updated iproute2 or source patching) and a proper QoS configuration on your network.

The QoS configuration is up to you. However, CoDel is a improvement regardless of your setup.

In case you're feeling adventurous AND you know what you're doing. You can always tweak the configuration inside the source code. Follows the configuration I'm using for my ADSL 5Mib D/ 512Kib U when on WIFI. This is what I'm using for my Synology NAS. We probably can find "optimal" configurations for our Android phones but I'd be trying that one for now.

Let me know if there is anything I can do to help.

Best regards,

Code:
--- sch_fq_codel_core.c.orig 2013-02-13 09:12:35.654488441 -0200
+++ sch_fq_codel_core.c 2013-02-13 09:14:58.884429036 -0200
@@ -351,7 +351,7 @@
q->cparams.ecn = !!nla_get_u32(tb[TCA_FQ_CODEL_ECN]);

if (tb[TCA_FQ_CODEL_QUANTUM])
- q->quantum = max(256U, nla_get_u32(tb[TCA_FQ_CODEL_QUANTUM]));
+ q->quantum = max(500U, nla_get_u32(tb[TCA_FQ_CODEL_QUANTUM]));

#if (LINUX_VERSION_CODE <= KERNEL_VERSION(2,6,39))
while (sch->q.qlen > q->limit) {
@@ -408,7 +408,7 @@
int i;

#if (LINUX_VERSION_CODE <= KERNEL_VERSION(2,6,39))
- q->limit = 10*1024;
+ q->limit = 600;
#else
sch->limit = 10*1024;
#endif
The Following 14 Users Say Thank You to lioux For This Useful Post: [ View ] Gift lioux Ad-Free
 
 
24th February 2013, 02:54 PM |#2  
ak's Avatar
Senior Member
Flag Ak Land Valley
Thanks Meter: 69,912
 
Donate to Me
More
Im testing right now and porting SCH_NET from linux stable branch ...
Thanks a lot for your great work.
The Following 5 Users Say Thank You to ak For This Useful Post: [ View ] Gift ak Ad-Free
24th February 2013, 05:31 PM |#3  
lioux's Avatar
OP Senior Member
Thanks Meter: 208
 
Donate to Me
More
Quote:
Originally Posted by anarkia1976

Im testing right now and porting SCH_NET from linux stable branch ...
Thanks a lot for your great work.

SCH code from Linux 3.3 onward is based on Byte Queue Limits (BQL) which breaks compatibility with previous kernel code. It tracks bytes queue lenght instead of packets. It's both a saner and a more precise measure since packet size varies with MTU amongst other things. So 1 packet does not mean the same for every installation.

You might be able to import the *sch* files and use them with the compat.ko kernel module.
The Following User Says Thank You to lioux For This Useful Post: [ View ] Gift lioux Ad-Free
26th February 2013, 10:47 AM |#4  
ak's Avatar
Senior Member
Flag Ak Land Valley
Thanks Meter: 69,912
 
Donate to Me
More
I have implemented it into my kernel ... how can i verify performance?
The Following 2 Users Say Thank You to ak For This Useful Post: [ View ] Gift ak Ad-Free
26th February 2013, 09:14 PM |#5  
lioux's Avatar
OP Senior Member
Thanks Meter: 208
 
Donate to Me
More
Quote:
Originally Posted by anarkia1976

I have implemented it into my kernel ... how can i verify performance?

Let's try fq_codel: it implements fair share across multiple flows using CoDel algorithm. You can notice it easily if you start an upload THEN simultaneously try to browse, use VoIP, play a network game... anything that would require low latency. It fights bufferbloat, read the top post for further information on it.

fq_codel+upload tests should yield results closer to the ones without an upload. fq_codel should help maintain better latency than any other scheduling algorithm. The upload speed rates should remain mostly unaffected by fq_codel.

The tests were conducted on a stock 4.2.2 maguro Galaxy Nexus with
latest AK 3.0.67+~ak.710.422.Cylon kernel. I did the tests between 01:00 and 03:00 local time (GMT-3).

Code:
[email protected]:/ # uname -a
Linux localhost 3.0.67+~ak.710.422.Cylon #1 SMP PREEMPT Wed Feb 27 06:54:36 CET 2013 armv7l GNU/Linux
Read the full post to gather an idea of what to expect. root access is required to perform these series of tests.

Here follows a simple upload + several ping/latency test guide. The ping tests are done against 187.7.117.32 (www.google.com).

1) Install the following applications from Play Store
1.1) Fing
1.2) Net Ping
1.3) Network Latency Checker
1.4) Terminal

2) Copy a huge file to your Galaxy Nexus. We will upload it later to create the upload test environment.

3) Connect your Galaxy Nexus to your WIFI network.

4) Make sure there is no other traffic on your network: no downloads, youtube, nothing.

5) Make sure fq_codel is not enabled.
5.1) Open Terminal
5.2) Type the following commands to make sure fq_codel is disabled

Code:
su
tc qdisc del dev wlan0 root fq_codel
tc -s qdisc
5.3) You should get a result similar to the following. The wlan0 qdisc should read pfifo_fast.

Code:
[email protected]:/ # su
[email protected]:/ # tc qdisc del dev wlan0 root fq_codel
Android does not support qdisc 'fq_codel'
[email protected]:/ # tc -s qdisc
Android does not support qdisc 'prio'
qdisc pfifo_fast 0: dev rmnet0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 2889806 bytes 15890 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
Android does not support qdisc 'prio'
qdisc pfifo_fast 0: dev p2p0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 8892 bytes 114 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
Android does not support qdisc 'prio'
qdisc pfifo_fast 0: dev wlan0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 25272 bytes 438 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
6) Let's stablish a baseline. We will conduct all tests without any traffic on your network. Furthermore, note that fq_codel is not enabled.

7) Start Fing
7.1) Click Gear Icon
7.1.1) Host Tools -> Ping -> 187.7.117.32
7.2) Copy the results somewhere

8) Start Net Ping
8.1) Settings
8.1.1) Packet Count -> 100
8.1.2) Statistics -> Enable
8.1.3) Click Back
8.2) Ping 187.7.117.32
8.3) Copy the results somewhere

9) Start Network Latency Checker
9.1) Select 100 request , 0s delay , DNS
9.2) Click Check
9.3) Copy the results somewhere
9.4) Select 100 request , 0s delay , HTTP
9.5) Click Check
9.6) Copy the results somewhere

10) On your Galaxy Nexus, add the huge file from step 2 to your Google Drive. The upload will begin immediatly. If the upload finishes during any test, remove the file from Google Drive, upload it again then restart the given test.

11) Repeat Fing test
11.1) Copy the results somewhere

12) Repeat Net Ping test
12.1) Copy the results somewhere

13) Repeat Network Latency Checker test
13.1) Copy the results somewhere

14) Enable fq_codel. If the upload finishes, remove the file from Google Drive and upload it again.
14.1) Open Terminal
14.2) Type the following commands to make sure fq_codel is disabled

Code:
su
tc qdisc add dev wlan0 root fq_codel
tc -s qdisc
14.3) You should get a result similar to the following. The wlan0 qdisc should read fq_codel.

Code:
[email protected]:/ # su
[email protected]:/ # tc qdisc del dev wlan0 root fq_codel
Android does not support qdisc 'fq_codel'
[email protected]:/ # tc -s qdisc
Android does not support qdisc 'prio'
qdisc pfifo_fast 0: dev rmnet0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 2889806 bytes 15890 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
Android does not support qdisc 'prio'
qdisc pfifo_fast 0: dev p2p0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 8892 bytes 114 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
Android does not support qdisc 'fq_codel'
qdisc fq_codel 8001: dev wlan0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 525997 bytes 6231 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
15) Repeat Fing test
15.1) Copy the results somewhere

16) Repeat Net Ping test
16.1) Copy the results somewhere

17) Repeat Network Latency Checker test
17.1) Copy the results somewhere

-----

For instance, here follows my personal results. The upload tests were conducted with a 150Mb upload to Google Drive as fast as my ADSL would allow it. My ADSL has an upload max speed rate of 512KiB.

1) Fing
1.1) Results with neither upload nor fq_codel

Code:
64 average ping
6% package loss
60 minimum ping
112 maximum ping
0 std dev ping
5 estimated hops
1.2) Results with upload but without fq_codel

Code:
226 average ping
2% package loss
68 minimum ping
580 maximum ping
2 std dev ping
5 estimated hops
1.3) Results with both upload and fq_codel

Code:
180 average ping
6% package loss
67 minimum ping
510 maximum ping
3 std dev ping
5 estimated hops
2) Net Ping
2.1) Results with neither upload nor fq_codel

Code:
187.7.117.32 min-max: 60.6-161ms
187.7.117.32 avg/stddev: 69.9/15.1ms
2.2) Results with upload but without fq_codel

Code:
187.7.117.32 min-max: 69.6-510ms
187.7.117.32 avg/stddev: 251/110ms
2.3) Results with both upload and fq_codel

Code:
187.7.117.32 min-max: 68.5-454ms
187.7.117.32 avg/stddev: 220/93.4ms
3) Network Latency Checker: 100 request , 0s delay , DNS
3.1) Results with neither upload nor fq_codel

Code:
Min: 166 Max: 9338  Avg: 554
3.2) Results with upload but without fq_codel

Code:
Min: 176 Max: 9306 Avg: 629
3.3) Results with both upload and fq_codel

Code:
Min: 82 Max: 9649  Avg: 554
4) Network Latency Checker: 100 request , 0s delay , HTTP
4.1) Results with neither upload nor fq_codel

Code:
Min: 260 Max: 3280 Avg: 342
4.2) Results with upload but without fq_codel

Code:
Min: 239 Max: 10021 Avg: 1017
4.3) Results with both upload and fq_codel

Code:
Min: 81 Max: 3477 Avg: 560
The Network Latency Checker results present fq_codel as the best possible option on all cases. This was unexpected. I repeated those tests several times but fq_codel was always the best. I'll have to further review the Network Latency Checker tests. Can you reproduce these "weird" results?

I can say that I am really satisfied with fq_codel on Android. It gave better performance than plain pfifo_fast with little extra cost.

You can also try the exact same tests using 3G instead of WIFI. Replacing wlan0 with rmnet0 on the aforementioned steps should be enough. Check which interface is UP when you're using radio with the "netcfg" command on Terminal.

I reached out to bufferbloat mailing list for suggestions on how to better test and tune the code for Android phones.

[Bloat] Testing fq_codel on Android Galaxy Nexus AK kernel

Anarkia, thanks a lot for your continuous work on improving the Galaxy Nexus experience.

Please, do not hesitate to suggest improvements or corrections to this post.
The Following 2 Users Say Thank You to lioux For This Useful Post: [ View ] Gift lioux Ad-Free
27th February 2013, 05:32 PM |#6  
lioux's Avatar
OP Senior Member
Thanks Meter: 208
 
Donate to Me
More
Re: [KERNEL][MODULE] CoDel kernel modules for Android (fighting bufferbloat)
Dave Täht from BufferBloat project has given the following reply. I'll try to elaborate a script based on his suggestions.

Quote:

Dear Mario:

I read over the thread, don't have the energy to join the forum... Please forward?

1) The patch in that thread increases the default minimum quantum to 500. It should, IMHO, be available go down to 64U, and I in general get better results with smaller quantums than the default. Although 64 is a too small, 256 is not bad.

2) Wifi had 4 queues, not one, it is interesting to setup one fq_codel queue per queue. The various forms of the debloat script do this.

3) "Codel" and "FQ_codel" should not be used synonomously. Codel is a drop strategy, great for controlling queue length. FQ codel combines flow queueing (which interleaves flows together, so, for example, a dns packet or a gaming packet leaps to the head of a queue) with codel, the combination of which seems to be really, really good, if you have minimal driver buffering.

4) The problem on wifi/3g/lte/etc is that there is so much extra buffering at the driver and hardware level that hooking up fq_codel to it is like shaking at the end of a very, very long hose. Some of these phone wifi chips are hooked up via a vastly overbuffered usb bus. Shake all you want at one end of the hose, not a lot will happen.

But: It might help a little and I'd love to know more.

It would help if people fiddling with this stuff would take a gander at talks by van jacobson, eric dumazet and myself on the subject.

http://www.bufferbloat.net/projects/...i/Bloat-videos

http://netseminar.stanford.edu/

Youtube Video - What's wrong with Wi-Fi? by Dave Täht

That said, I'm delighted people are making a start at working on android. I hope myself to get some time this pring to fiddle with android, and I loved seeing the documentation on how to patch in fq_codel on the referred thread. Learning how to hack on a new embedded OS is hard.


Simply starting to measure the available buffering in the stack on a given chipset would be good. You can do that by shortening the txqueue and trying an upload, while measuring the delay with ping. Then repeat with a longer txqueuelen, and you can bracket how much buffering lies below to a large extent.

The bufferbloat crowd has smashed excess buffering throughout the tcp, qdisc, and ethernet and ADSL portions of the stack over the past year. It would be grand to get some insight as to what else to smash. It's like wackamole, only more fun....

The Following User Says Thank You to lioux For This Useful Post: [ View ] Gift lioux Ad-Free
4th March 2013, 02:00 AM |#7  
lioux's Avatar
OP Senior Member
Thanks Meter: 208
 
Donate to Me
More
Quote:
Originally Posted by lioux

Dave Täht from BufferBloat project has given the following reply. I'll try to elaborate a script based on his suggestions.

Quote:

This is a cross post from bloat mailing list.

Further information gathering. I won't procced with any further testing. I am neither sure how to interpret the data I already have for wlan0 nor that I am using the proper tools for the job.

I look forward to your input. I would like to be able to gather good values for both quantum and limit. I want to set them appropriately for the radio and wifi connections. Since we don't have the properly patched tc command, I'll probably be hard coding this on the fq_codel module itself.

Are there recommendations? If a guideline is available/suggested, I'll try to find good average values for this particular phone.

The data I gathered currently follows:

- Android phone Samsung Galaxy Nexus GSM I9250 running stock Yakju 4.2.2 with AK kernel
- WIFI connected to LAN, ADSL 512KiB U, Phone Google Drive upload at full speed. No other traffic on the network
- The drivers are available in binary form on Google page.
- Quick look on the Galaxy Nexus at RootzWiki
- There is a teardown on IFixit of all components
- Radio: Intel XG626 Baseband Modem
- WiFi / Bluetooth module: Broadcom BCM4330
- Samsung SWB-B42 BT 4.0 Dual Band Wlan FM Tx/Rx. Chipworks says the module is actually manufactured by Murata, and houses a Broadcom BCM4330 die inside.

Code:
# uname -a
Linux localhost 3.0.67+~ak.710.422.Cylon #1 SMP PREEMPT Wed Feb 27 06:54:36 CET 2013 armv7l GNU/Linux
# netcfg
lo       UP                                   127.0.0.1/8   0x00000049
ifb0     DOWN                                   0.0.0.0/0   0x00000082
ifb1     DOWN                                   0.0.0.0/0   0x00000082
tunl0    DOWN                                   0.0.0.0/0   0x00000080
sit0     DOWN                                   0.0.0.0/0   0x00000080
ip6tnl0  DOWN                                   0.0.0.0/0   0x00000080
rmnet0   DOWN                                   0.0.0.0/0   0x00001090
rmnet1   DOWN                                   0.0.0.0/0   0x00001090
rmnet2   DOWN                                   0.0.0.0/0   0x00001090
p2p0     UP                                     0.0.0.0/0   0x00001003
wlan0    UP                                  10.0.0.101/25  0x00001043
# ls -l /sys/class/net/p2p0/
-r--r--r-- root     root         4096 2013-03-02 21:33 addr_assign_type
-r--r--r-- root     root         4096 2013-03-02 21:33 addr_len
-r--r--r-- root     root         4096 2013-03-02 21:33 address
-r--r--r-- root     root         4096 2013-03-02 21:33 broadcast
-r--r--r-- root     root         4096 2013-03-02 21:33 carrier
-r--r--r-- root     root         4096 2013-03-02 21:33 dev_id
lrwxrwxrwx root     root              2013-03-02 21:33 device -> ../../../mmc1:0001:2
-r--r--r-- root     root         4096 2013-03-02 21:33 dormant
-r--r--r-- root     root         4096 2013-03-02 21:33 duplex
-r--r--r-- root     root         4096 2013-03-02 21:33 features
-rw-r--r-- root     root         4096 2013-03-02 21:33 flags
-rw-r--r-- root     root         4096 2013-03-02 21:33 ifalias
-r--r--r-- root     root         4096 2013-03-02 21:33 ifindex
-r--r--r-- root     root         4096 2013-03-02 21:33 iflink
-r--r--r-- root     root         4096 2013-03-02 21:33 link_mode
-rw-r--r-- root     root         4096 2013-03-02 21:33 mtu
-rw-r--r-- root     root         4096 2013-03-02 21:33 netdev_group
-r--r--r-- root     root         4096 2013-03-02 21:33 operstate
lrwxrwxrwx root     root              2013-03-02 21:33 phy80211 -> ../../ieee80211/phy0
drwxr-xr-x root     root              2013-03-02 21:33 power
drwxr-xr-x root     root              2013-03-02 21:33 queues
-r--r--r-- root     root         4096 2013-03-02 21:33 speed
drwxr-xr-x root     root              2013-03-02 21:33 statistics
lrwxrwxrwx root     root              2013-03-02 21:33 subsystem -> ../../../../../../../../../../class/net
-rw-r--r-- root     root         4096 2013-03-02 21:33 tx_queue_len
-r--r--r-- root     root         4096 2013-03-02 21:33 type
-rw-r--r-- root     root         4096 2013-03-02 21:33 uevent

# ls -l /sys/class/net/wlan0/
-r--r--r-- root     root         4096 2013-03-02 21:38 addr_assign_type
-r--r--r-- root     root         4096 2013-03-02 21:38 addr_len
-r--r--r-- root     root         4096 2013-03-02 21:38 address
-r--r--r-- root     root         4096 2013-03-02 21:38 broadcast
-r--r--r-- root     root         4096 2013-03-02 21:38 carrier
-r--r--r-- root     root         4096 2013-03-02 21:38 dev_id
lrwxrwxrwx root     root              2013-03-02 21:38 device -> ../../../mmc1:0
001:2
-r--r--r-- root     root         4096 2013-03-02 21:38 dormant
-r--r--r-- root     root         4096 2013-03-02 21:38 duplex
-r--r--r-- root     root         4096 2013-03-02 21:38 features
-rw-r--r-- root     root         4096 2013-03-02 21:38 flags
-rw-r--r-- root     root         4096 2013-03-02 21:38 ifalias
-r--r--r-- root     root         4096 2013-03-02 21:38 ifindex
-r--r--r-- root     root         4096 2013-03-02 21:38 iflink
-r--r--r-- root     root         4096 2013-03-02 21:38 link_mode
-rw-r--r-- root     root         4096 2013-03-02 21:38 mtu
-rw-r--r-- root     root         4096 2013-03-02 21:38 netdev_group
-r--r--r-- root     root         4096 2013-03-02 21:38 operstate
lrwxrwxrwx root     root              2013-03-02 21:38 phy80211 -> ../../ieee80211/phy0
drwxr-xr-x root     root              2013-03-02 21:38 power
drwxr-xr-x root     root              2013-03-02 21:38 queues
-r--r--r-- root     root         4096 2013-03-02 21:38 speed
drwxr-xr-x root     root              2013-03-02 21:31 statistics
lrwxrwxrwx root     root              2013-03-02 21:38 subsystem -> ../../../../../../../../../../class/net
-rw-r--r-- root     root         4096 2013-03-02 21:38 tx_queue_len
-r--r--r-- root     root         4096 2013-03-02 21:38 type
-rw-r--r-- root     root         4096 2013-03-02 21:38 uevent
By quick traffic examination, it seems that all WIFI traffic flows through wlan0:

Code:
# tc -s qdisc
Android does not support qdisc 'fq_codel'
qdisc fq_codel 8001: dev rmnet0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
Android does not support qdisc 'prio'
qdisc pfifo_fast 0: dev p2p0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 468 bytes 6 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
Android does not support qdisc 'prio'
qdisc pfifo_fast 0: dev wlan0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 38442999 bytes 26737 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
Here follows the information on wlan0 interface:

Code:
# cat /sys/class/net/wlan0/tx_queue_len
1000
# cat /sys/class/net/wlan0/speed
sh: cat: /sys/class/net/wlan0/speed: Invalid argument
# ls -l /sys/class/net/wlan0/queues
drwxr-xr-x root     root              2013-03-02 22:04 rx-0
drwxr-xr-x root     root              2013-03-02 22:04 tx-0
# ls -l /sys/class/net/wlan0/queues/tx-0
-rw-r--r-- root     root         4096 2013-03-02 22:04 xps_cpus
Let's try to measure the hardware buffer by sending 500 pings to 187.7.117.32 with "Net Ping" from Play Store. I'll use different txqueuelen values each time. There is a concurrent upload at full speed to Google Drive.

Code:
# echo 0 > /sys/class/net/wlan0/tx_queue_len
# cat /sys/class/net/wlan0/tx_queue_len
0

500 pings
(187.7.117.32) min-max: 61.6-1058 ms
(187.7.117.32) avg/stddev: 315/167 ms

# echo 1 > /sys/class/net/wlan0/tx_queue_len

500 pings
(187.7.117.32) min-max: 60.3-1365 ms
(187.7.117.32) avg/stddev: 602/310 ms

# echo 10 > /sys/class/net/wlan0/tx_queue_len

500 pings
(187.7.117.32) min-max: 61.2-1337  ms
(187.7.117.32) avg/stddev: 571/288 ms

# echo 100 > /sys/class/net/wlan0/tx_queue_len

500 pings
(187.7.117.32) min-max: 61.2-1455  ms
(187.7.117.32) avg/stddev: 647/343 ms

# echo 1000 > /sys/class/net/wlan0/tx_queue_len

500 pings
(187.7.117.32) min-max: 60.0-1484  ms
(187.7.117.32) avg/stddev: 616/342 ms
Repeat the above tests without the concurrent upload.

Code:
# echo 0 > /sys/class/net/wlan0/tx_queue_len
# echo 0 > /sys/class/net/p2p0/tx_queue_len

500 pings
(187.7.117.32) min-max: 61.7-213 ms
(187.7.117.32) avg/stddev: 70.6/15.0 ms

# echo 1 > /sys/class/net/wlan0/tx_queue_len

500 pings
(187.7.117.32) min-max: 60.7-207 ms
(187.7.117.32) avg/stddev: 70.5/15.4 ms

# echo 10 > /sys/class/net/wlan0/tx_queue_len

500 pings
(187.7.117.32) min-max: 60.6-244  ms
(187.7.117.32) avg/stddev: 69.1/10.5 ms

# echo 100 > /sys/class/net/wlan0/tx_queue_len

500 pings
(187.7.117.32) min-max: 60.7-246  ms
(187.7.117.32) avg/stddev: 70.1/15.4 ms

# echo 1000 > /sys/class/net/wlan0/tx_queue_len

500 pings
(187.7.117.32) min-max: 61.4-251  ms
(187.7.117.32) avg/stddev: 70.2/16.3 ms
I've disabled WIFI and enabled 3G. I've got a HDSPA connection.

Code:
# netcfg
lo       UP                                   127.0.0.1/8   0x00000049
00:00:00
ifb0     DOWN                                   0.0.0.0/0   0x00000082
f9:64:69
ifb1     DOWN                                   0.0.0.0/0   0x00000082
c6:b4:f9
tunl0    DOWN                                   0.0.0.0/0   0x00000080
00:00:00
sit0     DOWN                                   0.0.0.0/0   0x00000080
00:00:00
ip6tnl0  DOWN                                   0.0.0.0/0   0x00000080
00:00:00
rmnet0   UP                              179.225.225.71/24  0x000010d1
00:00:00
rmnet1   DOWN                                   0.0.0.0/0   0x00001090
00:00:00
rmnet2   DOWN                                   0.0.0.0/0   0x00001090
00:00:00
p2p0     DOWN                                   0.0.0.0/0   0x00001002
c5:c2:4e
wlan0    DOWN                                   0.0.0.0/0   0x00001002
Here follows the information on rmnet0 interface:

Code:
# cat /sys/class/net/rmnet0/tx_queue_len
1000
# cat /sys/class/net/rmnet0/speed
/system/bin/sh: cat: /sys/class/net/rmnet0/speed: Invalid argument
# ls -l /sys/class/net/rmnet0/queues
drwxr-xr-x root     root              2013-03-03 15:03 rx-0
drwxr-xr-x root     root              2013-03-03 15:03 tx-0
# ls -l /sys/class/net/rmnet0/queues/tx-0
-rw-r--r-- root     root         4096 2013-03-03 15:03 xps_cpus
4th March 2013, 02:58 AM |#8  
lioux's Avatar
OP Senior Member
Thanks Meter: 208
 
Donate to Me
More
Quote:
Originally Posted by anarkia1976

I have implemented it into my kernel ... how can i verify performance?

You should be able following my guide below. A couple things though:

1) fq_codel is tuned to Gig networking per default.
1.1) The following patch should alleviate to our sub 10Mbit network (radio) without much effect on our WIFI traffic

Code:
diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c
index a7b3754..c765e6c 100644
--- a/net/sched/sch_fq_codel.c
+++ b/net/sched/sch_fq_codel.c
@@ -387,7 +387,7 @@ static int fq_codel_init(struct Qdisc *sch, struct nlattr *opt)
        struct fq_codel_sched_data *q = qdisc_priv(sch);
        int i;
 
-       sch->limit = 10*1024;
+       sch->limit = 600;
        q->flows_cnt = 1024;
        q->quantum = psched_mtu(qdisc_dev(sch));
        q->perturbation = net_random();
2) From my extensive testing, we should be safe to

# echo 0 > /sys/class/net/p2p0/tx_queue_len
# echo 0 > /sys/class/net/rmnet0/tx_queue_len
# echo 0 > /sys/class/net/wlan0/tx_queue_len

3) In case, you've codel the default instead of pfifo_fast... just make sure you've chosen fq_codel. That's the apropriate one. If not, we still need a startup script that would do

# tc qdisc add dev p2p0 root fq_codel
# tc qdisc add dev rmnet0 root fq_codel
# tc qdisc add dev wlan0 root fq_codel
# echo 0 > /sys/class/net/p2p0/tx_queue_len
# echo 0 > /sys/class/net/rmnet0/tx_queue_len
# echo 0 > /sys/class/net/wlan0/tx_queue_len

I've been running this setup for the past 24 hours. Everything seems to be fine.
The Following User Says Thank You to lioux For This Useful Post: [ View ] Gift lioux Ad-Free
4th March 2013, 05:03 PM |#9  
ak's Avatar
Senior Member
Flag Ak Land Valley
Thanks Meter: 69,912
 
Donate to Me
More
Re: [KERNEL][MODULE] CoDel kernel modules for Android (fighting bufferbloat)
Thanks lisan for your hard work :thumbup:

Sent from my Galaxy Nexus using xda app-developers app
The Following User Says Thank You to ak For This Useful Post: [ View ] Gift ak Ad-Free
10th March 2013, 10:51 AM |#10  
lioux's Avatar
OP Senior Member
Thanks Meter: 208
 
Donate to Me
More
Quote:
Originally Posted by anarkia1976

Thanks lisan for your hard work :thumbup:

Sent from my Galaxy Nexus using xda app-developers app

For anyone wanting to test either codel or fq_codel, use the AK Cylon kernel:

http://forum.xda-developers.com/show...4#post38967264

They're both implemented and I've sent the developer TCP Small Queues patch which should be enabled after cylon version 733

http://git.kernel.org/cgit/linux/ker...5ca784e34786c5

AK is doing its part on being the lowest network latency kernel out there. Power to AK kernel.
The Following 2 Users Say Thank You to lioux For This Useful Post: [ View ] Gift lioux Ad-Free
30th April 2014, 02:30 AM |#11  
jawz101's Avatar
Senior Member
Thanks Meter: 700
 
More
I'm trying your tests with @mrg666 kernel for the Nexus 4 since he has codel in his kernel but my results were not conclusive.
HTML Code:
+----------------------+--------------------+--------------------------+-----------------------------+------------------------------+
|        CODEL         | FING avg ping (ms) | NETPING avg/std dev (ms) | NW LATENCY CKR DNS avg (ms) | NW LATENCY CKR HTTP avg (ms) |
+----------------------+--------------------+--------------------------+-----------------------------+------------------------------+
| DISABLED             |                328 | 304/50.1                 |                         243 |                          238 |
| DISABLED + UPLOADING |                333 | 354/98.8                 |                         257 |                          268 |
| ENABLED              |                293 | 329/52.4                 |                         253 |                          253 |
| ENABLED + UPLOADING  |                328 | 340/94.8                 |                         310 |                          296 |
+----------------------+--------------------+--------------------------+-----------------------------+------------------------------+
... though I don't know if this means anything to you but the results of these commands didn't look the same as yours:
Code:
[email protected]:/ # tc qdisc add dev wlan0 root fq_codel
tc qdisc add dev wlan0 root fq_codel
Android does not support qdisc 'fq_codel'
RTNETLINK answers: No such file or directory
2|[email protected]:/ # tc -s qdisc
tc -s qdisc
Android does not support qdisc 'prio'
qdisc pfifo_fast 0: dev rmnet_usb0 root refcnt 2 [cannot parse qdisc parameters]
 Sent 4224 bytes 36 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
Android does not support qdisc 'mq'
qdisc mq 0: dev wlan0 root 
 Sent 32731522 bytes 27551 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
Android does not support qdisc 'mq'
qdisc mq 0: dev p2p0 root 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0


---------- Post added at 08:30 PM ---------- Previous post was at 08:00 PM ----------

... maybe because he has codel enabled- not fq_codel... grr
Post Reply Subscribe to Thread

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes