[Think Tank] Cache DNS Requests on Android to speedup browsing

Search This thread

MrBlueEyez

Senior Member
Jul 30, 2009
67
0
Rotterdam
I was wondering if there is a method to cache DNS requests? I use CM latest and was browsing around in the system folder, when i came by /system/etc/resolv.conf. Here you find the nameservers used by android to resolve DNS.

This got me thinking, because I use a DNS cache on my Ubuntu Box. Might there be a way to do this on Android, to speed up browsing regularly visited websites? Does anyone have any idea how to use a local DNS cache to speed up browsing?

Here's the deal:

- In /system/etc/resolv.conf you find 4 nameservers (4.2.2.5 - 4.2.2.2)
- Doing a ping test (count = 1000) to those, i find big differences in latency. Average was 800 on my test on 3G (I live in the Netherlands). I found this number rather large.
- On IRC, when asking this question, they mentioned porting dnsmasq to android and change nameserver to localhost

A ported version of dnsmasq is found here: http://code.google.com/p/android-wifi-tether/source/browse/trunk/res/raw/dnsmasq?spec=svn120&r=120

Copied the file to /system/bin, chmod +x the file, and this one works.

Now here's the thing:

- dnsmasq --help mentions a configurationfile: /data/local/dnsmasq.conf. This one have to be made for android
- how to configure dnsmasq so it uses the DNS servers and create a cache, thus speeding up browsing (in my humble belief)
- how to start dnsmasq on booting android (and making sure it stays alive)

And finally, does it make sense to use this kind of method? Like to apply for a think tank to make this work
 

the_fish

Senior Member
May 12, 2008
611
4
That does sound like a good idea. For me it isn't just 800ms to find out the DNS but on edge rather something like about 3 seconds! this would seriously speed the whole thing up! However, how are you gonna find the IP adresses for the first time?
 

MrBlueEyez

Senior Member
Jul 30, 2009
67
0
Rotterdam
Well dnsmasq creates a cache, using nameservers found in the config file, if i am correct. I am looking into the dnsmasq.conf examples to see what workaround i need.. Should be pretty straightforward

resolv.conf should have nameserver 127.0.0.1, and dnsmasq should use configured nameservers like 4.2.2.5 and create a cache. Therefore apps use the dnsmasq as DNS server.

Furthermore, i am trying to find out what nameservers are the fastest. 4.2.2.2-5 are Verizon, while i am in the Netherlands. OpenDNS has a datacenter in Amsterdam, so i figured changing nameservers to OpenDNS would speed things up some more..
 

olearyp

Senior Member
Sep 1, 2009
1,161
151
Even better would be unbound, though the stated overhead of ~11 MB might be too much for G1/mT3G v1.

Also, do you have to do any tricks to prevent resolv.conf from being overwritten when you switch networks? I know that usually dhclient will rewrite the file, and there's a couple of workarounds, but I have no idea which ones would work on Android.
 
Last edited:

cyanogen

Retired Recognized Developer
Jan 4, 2009
2,515
5,752
Seattle
cyanogenmod.com
CM builds already have a dnsmasq binary in /system/bin. It's used for tethering. You could launch it with a different config file if needed (there's a dnsmasq.conf in /system/etc already for tethering). Also, the values in resolv.conf aren't really used by much anymore unless you have stuff linked with uclibc. The resolver in Bionic uses the values of the net.dns* system properties.
 

MrBlueEyez

Senior Member
Jul 30, 2009
67
0
Rotterdam
CM builds already have a dnsmasq binary in /system/bin. It's used for tethering. You could launch it with a different config file if needed (there's a dnsmasq.conf in /system/etc already for tethering). Also, the values in resolv.conf aren't really used by much anymore unless you have stuff linked with uclibc. The resolver in Bionic uses the values of the net.dns* system properties.

Thanks for the reaction... You are right indeed, i found the dnsmasq.conf:
Code:
no-resolv
no-poll
server=4.2.2.2
server=4.2.2.3
interface=usb0
dhcp-authoritative 
dhcp-range=192.168.77.100,192.168.77.105,12h
dhcp-leasefile=/data/misc/dnsmasq.leases
pid-file=/data/misc/dnsmasq.pid
user=dhcp
no-negcache

What you're saying is that i should create another one, but i am wondering what values i should use..

interface=?

Also, regarding your remark on DNS, how to see the values of net.dns* and how to change them? sysctl -n doesn't show these values, I must be looking in the wrong direction...
 
Last edited:

zrubi

Member
Dec 14, 2008
42
0
Hey,

You don't have to set an interface at all.
The interface delcaretion limit dnsmasq to listen only on a specific interface (for both DHCP and DNS requests).
As far as I know dnsmasq is caching dns by default you can limit the cache size and set not to cache negative queries but it will cache by default so no special settings is needed.

In addition, Dnsmasq is also function as dhcp server and if you don't want it to try removing all lines that declaring on dhcp settings.
 
Last edited:

MrBlueEyez

Senior Member
Jul 30, 2009
67
0
Rotterdam
True on that one, i want to use dnsmasq mainly on 3G

On my remark about the dns properties, already found out to use getprop and setprop

getprop shows different values on DNS compared to resolv.conf:

[net.dns1.195]: [84.241.226.140]
[net.dns2.195]: [84.241.226.9]
[net.dns1]: [84.241.226.140]
[net.dns2]: [84.241.226.9]
 
Last edited:

lbcoder

Senior Member
Jan 21, 2009
2,613
98
You need to be aware of one thing with regards to the resolv.conf file.
It is NOT USED BY ANDROID.

If you use the command "getprop", you will see several dns entries in there -- none of which matches the static dns servers set in resolv.conf. The resolv.conf is used by TERMINAL BINARIES.
 

Firerat

Senior Member
Feb 24, 2009
3,848
185
Hey,

You don't have to set an interface at all.
The interface delcaretion limit dnsmasq to listen only on a specific interface (for both DHCP and DNS requests).
As far as I know dnsmasq is caching dns by default you can limit the cache size and set not to cache negative queries but it will cache by default so no special settings is needed.

In addition, Dnsmasq is also function as dhcp server and if you don't want it to try removing all lines that declaring on dhcp settings.

might as well restrict it to the local interface
Code:
interface=lo
you can then turn off dhcp with
Code:
no-dhcp-interface=lo
 

thebluecoat

Senior Member
Aug 7, 2009
204
74
I use OpenDNS on my home network which is claimed to be better than ISP dns servers. When you signup/setup you are required to input their values for dns servers, perhaps you set these values from your phone?
 

equid0x

Senior Member
Jan 18, 2009
60
15
Not sure if Android supports this... but the easiest and least resource intensive way to do this would be to simply add your commonly used domains to the hosts file. I believe some ad blocking software used to use this to block ads.

Aside from that... Bind can be run as a caching nameserver and would probably work in conjunction with setprop to overwrite the nameserver values android tries to use...

Another interesting idea might be to run a squid cache with data stored on the SD card... this should theoretically be faster than pulling the pages over 3G, but could be memory constrained.

FWIW I did some testing with timing page loads over 3G/2G/Wifi... and I found that the bottleneck for page loads on the G1 was not in fact the speed of the network connection(3G and WiFi were virtually identical - 12Mbps cable, 2G slightly slower), but CPU time spent in rendering(well, possibly memory related as well, but the G1 is always memory constrained so its hard to tell). The biggest speed improvement I experienced was in turning off Javascript.

I also tested Stock vs Dolphin vs Steel and found Stock & Steel to have very similar numbers, with Steel having a slight edge sometimes. Dolphin was always orders of magnitude slower.

IMHO The only real way to speed up browsing on the G1 would be to utilize an external compression proxy to reduce the amount of data being sent across the network to the G1, or to rewrite the browser/Dalvik VM/whatever to be more efficient. I tried the external proxy method but couldn't seem to get it to work.
 

MrBlueEyez

Senior Member
Jul 30, 2009
67
0
Rotterdam
@equid0x Good thoughts... I used Opera on Android, which uses a compression proxy. Runs fast. Might be an idea to look into this for the native browser and abandon the idea to cache DNS.
 

alapapa

Senior Member
Dec 18, 2007
456
8
CM builds already have a dnsmasq binary in /system/bin. It's used for tethering. You could launch it with a different config file if needed (there's a dnsmasq.conf in /system/etc already for tethering). Also, the values in resolv.conf aren't really used by much anymore unless you have stuff linked with uclibc. The resolver in Bionic uses the values of the net.dns* system properties.

I think I have dnsmasq playing nice w/ this config:
Code:
no-resolv
no-poll
server=4.2.2.2
server=4.2.2.3
listen-address=127.0.0.7
interface=lo
pid-file=/data/misc/dnsmasqcache.pid
user=dhcp
no-negcache

it's listening on UDP port 53 -- seems to be doing its thing..

you're right..resolv.conf doesn't affect anything I do.

but I don't see any entries like net.dns* when i
#sysctl net

What am I missing about how Bionic does things?
 

alapapa

Senior Member
Dec 18, 2007
456
8

Thanks.

i can:
# setprop net.dns1 127.0.0.1
# setprop net.dns2 127.0.0.1

and it uses dnsmasq for a while, then they change back to t-mobile's (seems like it happens when i change towers or it goes from 3g->edge or vice-versa

does net.dnschange have any ability to control this?
know where I can find any documentation on the dhcp process that populates these values?

edit: just tested again and the settings persisted all night. performance wasn't noticeably different than normal. I'll try to do some more scientific tests and report back..
 
Last edited:

zioalex

Member
Jun 3, 2009
25
0
dnsmasq cpu problem

Hi Guys,
I've implemented the dnsmasq with my adhoc connection. My config file is:
no-resolv
no-poll
server=10.50.30.254
listen-address=127.0.0.1
interface=lo
pid-file=/data/misc/dnsmasqcache.pid
user=dhcp
no-negcache
addn-hosts=/data/misc/dnsmasq-host
cache-size=65536
local-ttl=86400

where the dnsmasq-host file is a copy of http://www.mvps.org/winhelp2002/hosts.txt to avoid advertising site.

After few minutes I run dnsmasq I get the cpu at 100% and the process sayd:
dnsmasq: Unknown cmd ''
dnsmasq: Unknown cmd ''
dnsmasq: Unknown cmd ''

I've tried only with few config options but the result is the same.

Do you know what is that?