Understanding Pry-Fi and 802.11 probe requests

Search This thread

cernekee

Senior Member
Jun 2, 2013
186
427
Over the past year or so, a number of news stories have discussed the possibility of tracking smartphone users in public via wifi probes. The recent announcement of Pry-Fi inspired me to do a little research to see exactly what information was being revealed, and what could be done about it.

The easiest way to observe the information leak in question (and later, to find out if it has been successfully plugged) is to follow part 4 of Vivek Ramachandran's WLAN Security Megaprimer. He describes the process of setting up a virtual machine running Backtrack Linux (now replaced by Kali Linux), and using aircrack-ng and wireshark to observe WLAN activity.

The whole series is worth watching, but here are a few basic survival commands:

Code:
# enable RF monitor mode on wlan0 (your interface might have a different name)
airmon-ng start wlan0

# compile a list of known networks and their BSSIDs; hit ^C to quit
airodump-ng mon0

# tune the WLAN card to the channel of interest (seen in the above list)
iwconfig wlan0 channel 1

# start wireshark and then begin sniffing
wireshark

Android uses the standard wpa_supplicant infrastructure found on Linux PCs to manage the WLAN interface and its database of known access points (APs). The configuration is stored in /data/misc/wifi/wpa_supplicant.conf. I was not able to find this mirrored anywhere else (e.g. in a SQLite database).

The wpa_supplicant daemon is controlled through commands sent over its standard socket interface by $AOSP/frameworks/base/wifi/java/android/net/wifi/WifiInfo.java, its JNI helpers, and libhardware. If your ROM supports it, you can observe what wpa_supplicant is doing by running wpa_cli from a shell, then in another window, running 'adb logcat -s "wpa_supplicant:*"'. Some useful wpa_cli commands include:

Code:
# show each outbound message (other levels include: info, debug, excessive)
log_level msgdump

# initiate a scan and dump results (note that the results also show up in
# logcat if the log_level is sufficiently high)
scan
scan_results

# show current status
status

# show known networks
list_networks

# disable scan_ssid on entry #2 from list_networks
set_network 2 scan_ssid 0
save

# reread the wpa_supplicant.conf file
reconfigure

On my N5 running KK 4.4 I needed to preface most commands with "IFNAME=wlan0", but on my N7 running JB 4.2.2 only the bare commands were accepted. YMMV so try both if in doubt.

Here are some observations gathered from experimenting with Nexus 7 (2012) running JB 4.2.2:

  • There is an entry in wpa_supplicant.conf for every known network.
  • Any network added manually with the '+' icon will have scan_ssid=1. Any network picked from the list of scan results will not have a scan_ssid property.
  • When wifi is enabled and the user is in the Settings->Wifi activity, Android will perform a network scan about every 8 seconds.
  • When wifi is enabled and the user is doing something else, and the interface is not currently associated with an AP, Android will perform a network scan about every 15 seconds.
  • On each scan, wpa_supplicant will send out probe requests. Every probe request contains the interface's MAC address. :(
  • wpa_supplicant will send probe requests containing an explicit ESSID name for each entry that has scan_ssid=1. If there are no such entries, the SSID parameter in the probe request will always be empty (signifying a broadcast probe request).
  • If wpa_supplicant.conf contains a very common ESSID name (like "linksys" or "netgear") then your device will try to associate with any other AP that uses that name. i.e. as expected, it matches the ASCII ESSID rather than the BSSID MAC address
  • If your device associates with a network, it leaks a whole bunch more data (and could be victimized by a malicious network). So if privacy/security are important, you'll want to carefully control the conditions under which your device associates with wifi networks.
  • N.B. The baseband side might not be any more trustworthy, but that is beyond the scope of this discussion.

The latter two items potentially leak sensitive data: your MAC address uniquely identifies your smartphone, and an ESSID list can reveal networks to which you have connected in the past (home networks, work networks, etc). In some cases the network names may be unique enough to translate back to a specific geographical location. The ESSID is just an ASCII string, not the AP's MAC address, so it may introduce some degree of ambiguity.

If you're monitoring wpa_supplicant via wpa_cli and logcat, and you used "log_level msgdump", you can see which ESSID names your device is broadcasting:

Code:
D/wpa_supplicant(19967): wlan0: Control interface command 'SCAN'
D/wpa_supplicant(19967): wlan0: Setting scan request: 0 sec 0 usec
[color=red]D/wpa_supplicant(19967): Scan SSID - hexdump(len=7): 61 74 74 77 69 66 69[/color]
D/wpa_supplicant(19967): wlan0: Starting AP scan for wildcard SSID
D/wpa_supplicant(19967): WPS: Building WPS IE for Probe Request
D/wpa_supplicant(19967): WPS:  * Version (hardcoded 0x10)
D/wpa_supplicant(19967): WPS:  * Request Type
D/wpa_supplicant(19967): WPS:  * Config Methods (4388)
D/wpa_supplicant(19967): WPS:  * UUID-E
D/wpa_supplicant(19967): WPS:  * Primary Device Type
D/wpa_supplicant(19967): WPS:  * RF Bands (1)
D/wpa_supplicant(19967): WPS:  * Association State
D/wpa_supplicant(19967): WPS:  * Configuration Error (0)
D/wpa_supplicant(19967): WPS:  * Device Password ID (0)
D/wpa_supplicant(19967): WPS:  * Manufacturer
D/wpa_supplicant(19967): WPS:  * Model Name
D/wpa_supplicant(19967): WPS:  * Model Number
D/wpa_supplicant(19967): WPS:  * Device Name
D/wpa_supplicant(19967): WPS:  * Version2 (0x20)
D/wpa_supplicant(19967): P2P: * P2P IE header
D/wpa_supplicant(19967): P2P: * Capability dev=24 group=00
D/wpa_supplicant(19967): P2P: * Listen Channel: Regulatory Class 81 Channel 6
[color=red]D/wpa_supplicant(19967): nl80211: Scan SSID - hexdump(len=7): 61 74 74 77 69 66 69[/color]
[color=blue]D/wpa_supplicant(19967): nl80211: Scan SSID - hexdump(len=0): [NULL][/color]

The red lines show probe requests that leak the "attwifi" ESSID and your MAC address. The blue line shows a probe request with an empty (broadcast) ESSID, which still leaks your MAC address.

So, what can we do about these leaks? I'll post a list of suggestions and see what else the XDA community comes up with:

  • Check your wpa_supplicant.conf for any entries with scan_ssid=1. If the network has SSID broadcasts disabled, you need this property; if you are the administrator you can re-enable SSID broadcasts on the AP and then eliminate scan_ssid=1. Vivek's videos explain why disabling SSID broadcast on the AP does not actually provide any security benefits.
  • Disable "Keep Wi-Fi on during sleep." This option is found under Settings -> Wi-Fi -> (menu) -> Advanced. If wifi is disabled while the phone is in your pocket sleeping, it will not broadcast its MAC address every 15 seconds. This isn't a comprehensive solution but it reduces the scope of the problem.
  • Run Smarter Wi-Fi Manager. SWM uses your coarse (cellular/GPS?) location to decide whether or not to enable the WLAN radio. Pros: open source; eliminates many possible information leaks. Cons: I believe this requires location services enabled, which may supply location data to other apps and/or impact battery life.
  • Run Pry-Fi. Pros: I have verified through Wireshark that this indeed randomizes the MAC addresses in the Probe Request packets. Cons: Not open source (and it's a "security" app that runs as root); causes strange interactions with the standard Android UI. Other potential side effects are unknown as the code has not yet been analyzed.
  • Modify the ROM. It may be possible to tweak wpa_supplicant or the kernel WLAN driver to send random or generic (ff:ff:ff:ff:ff:ff?) MAC addresses in their probe requests. For wpa_supplicant this may be doable through Cydia Substrate. It is likely that a well-done ROM modification would have few or no side effects on usability. However, it may require each individual wpa_supplicant driver or kernel WLAN driver to be modified.
  • Modify the UI. It would be nice if the Settings -> Wi-Fi interface clearly distinguished entries with scan_ssid=1. Maybe these could be shown in red with a simple Xposed module or ROM tweak.

Any other ideas?
 
Last edited:

kenshin33

Senior Member
Mar 25, 2009
333
133
Montreal
  • Check your wpa_supplicant.conf for any entries with scan_ssid=1. If the network has SSID broadcasts disabled, you need this property; if you are the administrator you can re-enable SSID broadcasts on the AP and then eliminate scan_ssid=1. Vivek's videos explain why disabling SSID broadcast does not actually provide any security benefits.


  • Modify the ROM. It may be possible to tweak wpa_supplicant or the kernel WLAN driver to send random or generic (ff:ff:ff:ff:ff:ff?) MAC addresses in their probe requests. For wpa_supplicant this may be doable through Cydia Substrate. It is likely that a well-done ROM modification would have few or no side effects on usability. However, it may require each individual wpa_supplicant driver or kernel WLAN driver to be modified.

Wha do you mean by: " tweak wpa_supplicant or the kernel WLAN driver to send random or generic (ff:ff:ff:ff:ff:ff?) MAC addresses in their probe request" ?
just the prob request??
Some devices (Galaxy nexus -torro in particular-) have already that, each boot gives a random mac address (unless specified as a boot arg).
But this is not an ideal solution b/c it can bring other problems (exhaust dhcp pools, duplicate macs on same network ..... etc).
The same problem would apply to anyone with a laptop and frequents regularly some coffe shop or library ...


(I use LLama -doesn't need location service on- to enable/disable Wifi)
 

cernekee

Senior Member
Jun 2, 2013
186
427
Wha do you mean by: " tweak wpa_supplicant or the kernel WLAN driver to send random or generic (ff:ff:ff:ff:ff:ff?) MAC addresses in their probe request" ?
just the prob request??

Well, the main problem (from my standpoint at least) is that any time wifi is enabled, the device will broadcast its MAC address. Even if there aren't any familiar networks in range. This is what allows for surreptitious tracking.

So if the probe request is either eliminated, or modified to avoid giving out identification information, this problem is avoided.

Some devices (Galaxy nexus -torro in particular-) have already that, each boot gives a random mac address (unless specified as a boot arg).
But this is not an ideal solution b/c it can bring other problems (exhaust dhcp pools, duplicate macs on same network ..... etc).

Also, a production device might not be rebooted very often.

A reasonable tradeoff might involve changing to a randomized MAC address when the wifi device is brought up, possibly rate-limited to once or twice a day. This could potentially be done by wrapping wpa_supplicant with a script (or just hacking the source code).

There are a couple of MAC address changer apps on Google Play, but unfortunately nothing on F-Droid. Obviously this operation requires root.

Edit: RandoMAC may be a starting point

Another option might involve adding a "change MAC address every N hours" feature to an existing app, such as AFWall.

The same problem would apply to anyone with a laptop and frequents regularly some coffe shop or library ...

If you're actually passing data traffic on a public wifi service, it becomes significantly harder to prevent information leaks and avoid leaving traces of your presence.

FWIW I noticed that mac80211 in the kernel will perform a passive scan if no SSIDs are supplied by the caller:

Code:
		if ((req->channels[0]->flags &
		     IEEE80211_CHAN_PASSIVE_SCAN) ||
		    !local->scan_req->n_ssids) {
			next_delay = IEEE80211_PASSIVE_CHANNEL_TIME;
		} else {
			ieee80211_scan_state_send_probe(local, &next_delay);
			next_delay = IEEE80211_CHANNEL_TIME;
		}

So I tried modifying driver_nl80211.c in wpa_supplicant so that it passes in 0 SSIDs, 0 extra IEs, and 0 frequencies. Yet the bcmdhd kernel driver (which implements a "full mac" in firmware, rather than using mac80211) still sent the probe requests.

wpa_supplicant / wpa_cli also recognizes a "DRIVER PASSIVE-SCAN" command which does not seem to have an effect.
 
Last edited:

kenshin33

Senior Member
Mar 25, 2009
333
133
Montreal
Well, the main problem (from my standpoint at least) is that any time wifi is enabled, the device will broadcast its MAC address. Even if there aren't any familiar networks in range. This is what allows for surreptitious tracking.

So if the probe request is either eliminated, or modified to avoid giving out identification information, this problem is avoided.



Also, a production device might not be rebooted very often.

A reasonable tradeoff might involve changing to a randomized MAC address when the wifi device is brought up, possibly rate-limited to once or twice a day. This could potentially be done by wrapping wpa_supplicant with a script (or just hacking the source code).

There are a couple of MAC address changer apps on Google Play, but unfortunately nothing on F-Droid. Obviously this operation requires root.

Edit: RandoMAC may be a starting point

Another option might involve adding a "change MAC address every N hours" feature to an existing app, such as AFWall.



If you're actually passing data traffic on a public wifi service, it becomes significantly harder to prevent information leaks and avoid leaving traces of your presence.

FWIW I noticed that mac80211 in the kernel will perform a passive scan if no SSIDs are supplied by the caller:

Code:
		if ((req->channels[0]->flags &
		     IEEE80211_CHAN_PASSIVE_SCAN) ||
		    !local->scan_req->n_ssids) {
			next_delay = IEEE80211_PASSIVE_CHANNEL_TIME;
		} else {
			ieee80211_scan_state_send_probe(local, &next_delay);
			next_delay = IEEE80211_CHANNEL_TIME;
		}

So I tried modifying driver_nl80211.c in wpa_supplicant so that it passes in 0 SSIDs, 0 extra IEs, and 0 frequencies. Yet the bcmdhd kernel driver (which implements a "full mac" in firmware, rather than using mac80211) still sent the probe requests.

wpa_supplicant / wpa_cli also recognizes a "DRIVER PASSIVE-SCAN" command which does not seem to have an effect.

wl_android.c doesn't seem to do anything with it
there are some refs in wl_cfg80211.c
 

cernekee

Senior Member
Jun 2, 2013
186
427
wl_android.c doesn't seem to do anything with it
there are some refs in wl_cfg80211.c

Well, this part of wl_android_priv_cmd() looks discouraging:

Code:
	else if (strnicmp(command, CMD_SCAN_ACTIVE, strlen(CMD_SCAN_ACTIVE)) == 0) {
		/* TBD: SCAN-ACTIVE */
	}
	else if (strnicmp(command, CMD_SCAN_PASSIVE, strlen(CMD_SCAN_PASSIVE)) == 0) {
		/* TBD: SCAN-PASSIVE */
	}

Also, I may have missed something obvious, but it isn't clear to me that wl->active_scan can ever get set to false:

Code:
$ git grep active_scan drivers/net/wireless/bcmdhd | cat
drivers/net/wireless/bcmdhd/wl_cfg80211.c:	passive_scan = wl->active_scan ? 0 : 1;
drivers/net/wireless/bcmdhd/wl_cfg80211.c:		err = wl_cfgp2p_escan(wl, ndev, wl->active_scan, num_chans, default_chan_list,
drivers/net/wireless/bcmdhd/wl_cfg80211.c:	passive_scan = wl->active_scan ? 0 : 1;
drivers/net/wireless/bcmdhd/wl_cfg80211.c:		passive_scan = wl->active_scan ? 0 : 1;
drivers/net/wireless/bcmdhd/wl_cfg80211.c:	beacon_proberesp = wl->active_scan ?
drivers/net/wireless/bcmdhd/wl_cfg80211.c:	wl->active_scan = true;
drivers/net/wireless/bcmdhd/wl_cfg80211.h:	bool active_scan;	/* current scan mode */
drivers/net/wireless/bcmdhd/wl_iw.c:wl_iw_set_active_scan(
drivers/net/wireless/bcmdhd/wl_iw.c:			ret = wl_iw_set_active_scan(dev, info, (union iwreq_data *)dwrq, extra);
drivers/net/wireless/bcmdhd/wl_iw.c:	(iw_handler)wl_iw_set_active_scan,

I think this line just sends a message to the firmware, so it isn't clear what is happening under the hood:

Code:
dev_wlc_ioctl(dev, WLC_SET_PASSIVE_SCAN, &as, sizeof(as));

The other bcmdhd mystery is how the MAC address gets set. When the interface is down, after a fresh boot, it has a fixed MAC address. But as soon as I run "ifconfig wlan0 up" it gets populated with the real MAC address. Changing the MAC address at runtime sometimes seems to stick; other times it doesn't.
 

kenshin33

Senior Member
Mar 25, 2009
333
133
Montreal
Well, this part of wl_android_priv_cmd() looks discouraging:

Code:
	else if (strnicmp(command, CMD_SCAN_ACTIVE, strlen(CMD_SCAN_ACTIVE)) == 0) {
		/* TBD: SCAN-ACTIVE */
	}
	else if (strnicmp(command, CMD_SCAN_PASSIVE, strlen(CMD_SCAN_PASSIVE)) == 0) {
		/* TBD: SCAN-PASSIVE */
	}

Also, I may have missed something obvious, but it isn't clear to me that wl->active_scan can ever get set to false:

Code:
$ git grep active_scan drivers/net/wireless/bcmdhd | cat
drivers/net/wireless/bcmdhd/wl_cfg80211.c:	passive_scan = wl->active_scan ? 0 : 1;
drivers/net/wireless/bcmdhd/wl_cfg80211.c:		err = wl_cfgp2p_escan(wl, ndev, wl->active_scan, num_chans, default_chan_list,
drivers/net/wireless/bcmdhd/wl_cfg80211.c:	passive_scan = wl->active_scan ? 0 : 1;
drivers/net/wireless/bcmdhd/wl_cfg80211.c:		passive_scan = wl->active_scan ? 0 : 1;
drivers/net/wireless/bcmdhd/wl_cfg80211.c:	beacon_proberesp = wl->active_scan ?
drivers/net/wireless/bcmdhd/wl_cfg80211.c:	wl->active_scan = true;
drivers/net/wireless/bcmdhd/wl_cfg80211.h:	bool active_scan;	/* current scan mode */
drivers/net/wireless/bcmdhd/wl_iw.c:wl_iw_set_active_scan(
drivers/net/wireless/bcmdhd/wl_iw.c:			ret = wl_iw_set_active_scan(dev, info, (union iwreq_data *)dwrq, extra);
drivers/net/wireless/bcmdhd/wl_iw.c:	(iw_handler)wl_iw_set_active_scan,

I think this line just sends a message to the firmware, so it isn't clear what is happening under the hood:

Code:
dev_wlc_ioctl(dev, WLC_SET_PASSIVE_SCAN, &as, sizeof(as));

The other bcmdhd mystery is how the MAC address gets set. When the interface is down, after a fresh boot, it has a fixed MAC address. But as soon as I run "ifconfig wlan0 up" it gets populated with the real MAC address. Changing the MAC address at runtime sometimes seems to stick; other times it doesn't.

seen it somewhere ... if you can find check one of the latest commit in anroid_kernel_samsung_tuna cm's github (they fixed the issus of random mac address each boot, they're still random but not that random).

I think in wl_cfg80211.c in one the init functions either one or the other is set (active/passive).
My guess is that the driver is on active scan by default an there's no "obvious" way of changing it, try fiddeling with those and see what happens
 

cernekee

Senior Member
Jun 2, 2013
186
427
seen it somewhere ... if you can find check one of the latest commit in anroid_kernel_samsung_tuna cm's github (they fixed the issus of random mac address each boot, they're still random but not that random).

Thanks for the pointer. It led me to this commit.

However, there is a difference between my platform (Nexus 7 2012) and their platform: they implement struct wifi_platform_data->get_mac_addr(). On the N7, dhd_custom_get_mac_address() returns -EOPNOTSUPP, so it goes off and queries the hardware for the MAC address:

Code:
#ifdef GET_CUSTOM_MAC_ENABLE
	ret = dhd_custom_get_mac_address(ea_addr.octet);
	if (!ret) {
		memset(buf, 0, sizeof(buf));
		bcm_mkiovar("cur_etheraddr", (void *)&ea_addr, ETHER_ADDR_LEN, buf, sizeof(buf));
		ret = dhd_wl_ioctl_cmd(dhd, WLC_SET_VAR, buf, sizeof(buf), TRUE, 0);
		if (ret < 0) {
			DHD_ERROR(("%s: can't set custom MAC address , error=%d\n", __FUNCTION__, ret));
			return BCME_NOTUP;
		}
		memcpy(dhd->mac.octet, ea_addr.octet, ETHER_ADDR_LEN);
	} else {
#endif /* GET_CUSTOM_MAC_ENABLE */
		/* Get the default device MAC address directly from firmware */
		memset(buf, 0, sizeof(buf));
		bcm_mkiovar("cur_etheraddr", 0, 0, buf, sizeof(buf));
		if ((ret = dhd_wl_ioctl_cmd(dhd, WLC_GET_VAR, buf, sizeof(buf),
			FALSE, 0)) < 0) {
			DHD_ERROR(("%s: can't get MAC address , error=%d\n", __FUNCTION__, ret));
			return BCME_NOTUP;
		}
		/* Update public MAC address after reading from Firmware */
		[b]memcpy(dhd->mac.octet, buf, ETHER_ADDR_LEN);[/b]
#ifdef GET_CUSTOM_MAC_ENABLE
	}
#endif /* GET_CUSTOM_MAC_ENABLE */

Adding printk()s to debug, the first time I see my real hardware address is at the memcpy(). Maybe this is set by the bootloader, or stored in an EEPROM, or something. Unfortunately I didn't see "cur_etheraddr" anywhere else in AOSP.

Another complicating factor: this function (dhd_preinit_ioctls()) runs every time the wlan0 device is opened. wpa_supplicant does the equivalent of an "ifconfig wlan0 up" when it starts. bcmdhd will accept a new MAC address when the interface is already up, but if you change the address while the interface is down, it will be reset back to the original address when it comes back up again. This is the opposite of most standard Linux network drivers.

A consequence of this quirk is that if the user does not check "Scanning always available", which keeps wpa_supplicant running in the background even when wifi is disabled, there is a race between Pry-Fi and wpa_supplicant. wpa_supplicant usually wins the race, leaking the device's true MAC address once before Pry-Fi kicks in.

I suspect that leaving the interface up all of the time will have a noticeable effect on battery life. And while users probably do not disable/enable wifi by hand too often, they might run other utilities that do so, expanding the window of vulnerability.

A more effective technique could involve hooking something like wpa_driver_nl80211_init() in wpa_supplicant so that it tries changing the MAC address before and after bringing up the interface (to cover both types of drivers).

Edit:

My guess is that the driver is on active scan by default an there's no "obvious" way of changing it, try fiddeling with those and see what happens

Setting wl->active_scan to false doesn't affect the result, nor does invoking "wldev_ioctl(dev, WLC_SET_PASSIVE_SCAN, &ps, sizeof(ps), 1)" with ps = 1 from wl_android.c.

wl_iw.c doesn't even get compiled into the kernel, so those ioctls probably won't help either.
 
Last edited:

kenshin33

Senior Member
Mar 25, 2009
333
133
Montreal
Thanks for the pointer. It led me to this commit.

However, there is a difference between my platform (Nexus 7 2012) and their platform: they implement struct wifi_platform_data->get_mac_addr(). On the N7, dhd_custom_get_mac_address() returns -EOPNOTSUPP, so it goes off and queries the hardware for the MAC address:

Code:
#ifdef GET_CUSTOM_MAC_ENABLE
	ret = dhd_custom_get_mac_address(ea_addr.octet);
	if (!ret) {
		memset(buf, 0, sizeof(buf));
		bcm_mkiovar("cur_etheraddr", (void *)&ea_addr, ETHER_ADDR_LEN, buf, sizeof(buf));
		ret = dhd_wl_ioctl_cmd(dhd, WLC_SET_VAR, buf, sizeof(buf), TRUE, 0);
		if (ret < 0) {
			DHD_ERROR(("%s: can't set custom MAC address , error=%d\n", __FUNCTION__, ret));
			return BCME_NOTUP;
		}
		memcpy(dhd->mac.octet, ea_addr.octet, ETHER_ADDR_LEN);
	} else {
#endif /* GET_CUSTOM_MAC_ENABLE */
		/* Get the default device MAC address directly from firmware */
		memset(buf, 0, sizeof(buf));
		bcm_mkiovar("cur_etheraddr", 0, 0, buf, sizeof(buf));
		if ((ret = dhd_wl_ioctl_cmd(dhd, WLC_GET_VAR, buf, sizeof(buf),
			FALSE, 0)) < 0) {
			DHD_ERROR(("%s: can't get MAC address , error=%d\n", __FUNCTION__, ret));
			return BCME_NOTUP;
		}
		/* Update public MAC address after reading from Firmware */
		[b]memcpy(dhd->mac.octet, buf, ETHER_ADDR_LEN);[/b]
#ifdef GET_CUSTOM_MAC_ENABLE
	}
#endif /* GET_CUSTOM_MAC_ENABLE */

Adding printk()s to debug, the first time I see my real hardware address is at the memcpy(). Maybe this is set by the bootloader, or stored in an EEPROM, or something. Unfortunately I didn't see "cur_etheraddr" anywhere else in AOSP.

Another complicating factor: this function (dhd_preinit_ioctls()) runs every time the wlan0 device is opened. wpa_supplicant does the equivalent of an "ifconfig wlan0 up" when it starts. bcmdhd will accept a new MAC address when the interface is already up, but if you change the address while the interface is down, it will be reset back to the original address when it comes back up again. This is the opposite of most standard Linux network drivers.

A consequence of this quirk is that if the user does not check "Scanning always available", which keeps wpa_supplicant running in the background even when wifi is disabled, there is a race between Pry-Fi and wpa_supplicant. wpa_supplicant usually wins the race, leaking the device's true MAC address once before Pry-Fi kicks in.

I suspect that leaving the interface up all of the time will have a noticeable effect on battery life. And while users probably do not disable/enable wifi by hand too often, they might run other utilities that do so, expanding the window of vulnerability.

A more effective technique could involve hooking something like wpa_driver_nl80211_init() in wpa_supplicant so that it tries changing the MAC address before and after bringing up the interface (to cover both types of drivers).

Edit:



Setting wl->active_scan to false doesn't affect the result, nor does invoking "wldev_ioctl(dev, WLC_SET_PASSIVE_SCAN, &ps, sizeof(ps), 1)" with ps = 1 from wl_android.c.

wl_iw.c doesn't even get compiled into the kernel, so those ioctls probably won't help either.
custom_get_mac_address doesnt'seem to do much
from grouper's :
Code:
int
dhd_custom_get_mac_address(unsigned char *buf)
{
        int ret = 0;

        WL_TRACE(("%s Enter\n", __FUNCTION__));
        if (!buf)
                return -EINVAL;

        /* Customer access to MAC address stored outside of DHD driver */
#if defined(CUSTOMER_HW2) && (LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 35))
        ret = wifi_get_mac_addr(buf);
#endif

#ifdef EXAMPLE_GET_MAC
        /* EXAMPLE code */
        {
                struct ether_addr ea_example = {{0x00, 0x11, 0x22, 0x33, 0x44, 0xFF}};
                bcopy((char *)&ea_example, buf, sizeof(struct ether_addr));
        }
#endif /* EXAMPLE_GET_MAC */

        return ret;
}
wifi_get_mac_addr(buf);
si an external function defined in wl_android.c

Code:
#if (LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 35))
int wifi_get_mac_addr(unsigned char *buf)
{
        DHD_ERROR(("%s\n", __FUNCTION__));
        if (!buf)
                return -EINVAL;
        if (wifi_control_data && wifi_control_data->get_mac_addr) {
                return wifi_control_data->get_mac_addr(buf);
        }
        return -EOPNOTSUPP;
}
#endif /* (LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 35)) */
as you can see it puts something in buf and returns -EOPNOTSUPP; no matter what.
the actual changing if possible is done with what''s inside the if

without recompiling the kernel try it ou with dhdutil
Code:
int
_dhd_set_mac_address(dhd_info_t *dhd, int ifidx, struct ether_addr *addr)
{
        char buf[32];
        wl_ioctl_t ioc;
        int ret;

        if (!bcm_mkiovar("cur_etheraddr", (char*)addr, ETHER_ADDR_LEN, buf, 32)) {
                DHD_ERROR(("%s: mkiovar failed for cur_etheraddr\n", dhd_ifname(&dhd->pub, ifidx)));
                return -1;
        }
        memset(&ioc, 0, sizeof(ioc));
        ioc.cmd = WLC_SET_VAR;
        ioc.buf = buf;
        ioc.len = 32;
        ioc.set = TRUE;

        ret = dhd_wl_ioctl(&dhd->pub, ifidx, &ioc, ioc.buf, ioc.len);
        if (ret < 0) {
                DHD_ERROR(("%s: set cur_etheraddr failed\n", dhd_ifname(&dhd->pub, ifidx)));
        } else {
                memcpy(dhd->iflist[ifidx]->net->dev_addr, addr, ETHER_ADDR_LEN);
                memcpy(dhd->pub.mac.octet, addr, ETHER_ADDR_LEN);
        }

        return ret;
}
that's from dhd_linux.c (under hardware/broadcom/ .... )
may be there's a way to do so with dhdutil (don't remeber all the possible things and both phone and tablet are far away )

EDIT : no dhdutil doesn't seem to have that !

the ioctl seems to end up here :

int
dhd_prot_ioctl(dhd_pub_t *dhd, int ifidx, wl_ioctl_t * ioc, void * buf, int len)
in dhd_cdc.c
 
Last edited:

cernekee

Senior Member
Jun 2, 2013
186
427
Code:
#if (LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 35))
int wifi_get_mac_addr(unsigned char *buf)
{
        DHD_ERROR(("%s\n", __FUNCTION__));
        if (!buf)
                return -EINVAL;
        if (wifi_control_data && wifi_control_data->get_mac_addr) {
                return wifi_control_data->get_mac_addr(buf);
        }
        return -EOPNOTSUPP;
}
#endif /* (LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 35)) */
as you can see it puts something in buf and returns -EOPNOTSUPP; no matter what.
the actual changing if possible is done with what''s inside the if

On tuna this should set the random (or not-so-random) MAC from rand_mac, then return 0:

https://github.com/CyanogenMod/andr...7e/arch/arm/mach-omap2/board-tuna-wifi.c#L288

But Tegra doesn't implement the get_mac_addr() callback, so bcmdhd will fall through to the -EOPNOTSUPP case and then query the firmware:

https://android.googlesource.com/ke...mr2/arch/arm/mach-tegra/board-grouper-sdhci.c
 

kenshin33

Senior Member
Mar 25, 2009
333
133
Montreal
On tuna this should set the random (or not-so-random) MAC from rand_mac, then return 0:

https://github.com/CyanogenMod/andr...7e/arch/arm/mach-omap2/board-tuna-wifi.c#L288

But Tegra doesn't implement the get_mac_addr() callback, so bcmdhd will fall through to the -EOPNOTSUPP case and then query the firmware:

https://android.googlesource.com/ke...mr2/arch/arm/mach-tegra/board-grouper-sdhci.c

you can may be copy it ?
(it doesn;t seem to do much except filling a buffer with data)

(as does the one from LGE msm for hammerhead, but reads from a file in /persiste/wifi/)
 

Chainfire

Moderator Emeritus / Senior Recognized Developer
Oct 2, 2007
11,452
87,856
www.chainfire.eu
  • When wifi is enabled and the user is in the Settings->Wifi activity, Android will perform a network scan about every 8 seconds.
  • When wifi is enabled and the user is doing something else, and the interface is not currently associated with an AP, Android will perform a network scan about every 15 seconds.

The background scan intervals are determined by these security settings (http://androidxref.com/4.4.2_r1/xref/frameworks/base/core/java/android/provider/Settings.java):

wifi_supplicant_scan_interval_ms
wifi_framework_scan_interval_ms

which in turn are sourced from these config values:

config_wifi_supplicant_scan_interval (15000 in AOSP)
config_wifi_framework_scan_interval (300000 in AOSP)

Inside the settings activity, it is determined by the WIFI_RESCAN_INTERVAL_MS field which is 10000 by default in AOSP (http://androidxref.com/4.4.2_r1/xre...ifi/WifiSettings.java#WIFI_RESCAN_INTERVAL_MS)

  • wpa_supplicant will send probe requests containing an explicit ESSID name for each entry that has scan_ssid=1. If there are no such entries, the SSID parameter in the probe request will always be empty (signifying a broadcast probe request).

I didn't see you mention this anywhere else, but the above doesn't appear to be entirely true, or is incomplete. On my test devices, there aren't any networks at all with scan_ssid=1, yet my MBA running Wireshark 24/7 (finally a use for a Mac) regularly intercepts the list of SSIDs (if Pry-Fi is not enabled). Certainly not as often as normal scans, but they're definitely still broadcast.

  • Check your wpa_supplicant.conf for any entries with scan_ssid=1. If the network has SSID broadcasts disabled, you need this property; if you are the administrator you can re-enable SSID broadcasts on the AP and then eliminate scan_ssid=1. Vivek's videos explain why disabling SSID broadcast does not actually provide any security benefits.

If you could quickly re-hash why there is no extra security in disabling SSID broadcasts, without having to review the entire Vivek video series, that'd be great.

  • Run Pry-Fi. Pros: I have verified through Wireshark that this indeed randomizes the MAC addresses in the Probe Request packets. Cons: Not open source (and it's a "security" app that runs as root); causes strange interactions with the standard Android UI. Other potential side effects are unknown as the code has not yet been analyzed.

It's not a security app, it's a privacy app, and you can't do this on a non-custom firmware without root, so I don't really see the point about complaining that its a security app that runs as root - most apps that do anything significant security or privacy wise do (requiring a framework/system modification and then not requesting root access is not an excuse, you still need root access for this at some point).

As for strange interactions, as documented, this is caused by the app having to change Wi-Fi states to let the MAC changes work. If it works as (currently) intended all you'd see is Wi-Fi going off/on again rapidly once or twice, and only if you are watching the settings->wifi screen. If anything else weird is going on, feel free to elaborate.

  • Run Smarter Wi-Fi Manager. SWM uses your coarse (cellular/GPS?) location to decide whether or not to enable the WLAN radio. Pros: open source; eliminates many possible information leaks. Cons: I believe this requires location services enabled, which may supply location data to other apps and/or impact battery life.

Have you actually reviewed this (and solutions like this) ? Do they even turn Wi-Fi fully off, including the background scanning ? AFAIK that's not possible without leveraging some form of root or system modification. And if it does uses that, it should be listed as a con because its a "security" app that runs as root. If it's a con for one app...

Also I would like to list as an additional con that this (fairly dramatically) reduces location determination speed and accuracy for location-based apps, which may or may not be an issue for the user.
 
  • Like
Reactions: zg85 and E:V:A

Chainfire

Moderator Emeritus / Senior Recognized Developer
Oct 2, 2007
11,452
87,856
www.chainfire.eu
FYI, I feel releasing source at this point is premature. I'm not really trying to hide anything here. If you really want the source right now I'll even give it to you (cernekee)

However, a lot of devices work in a slightly different way, and I wanted to get it out there to see how it would work out. After a lot of feedback, test versions, releases, etc (even though its only been 3 days) I feel a part of the core logic may need to be redesigned to incorporate the new knowledge (some even from this thread) to make it work better.

What I specifically don't want is a dozen or so source happy OSS zealots jumping on it, declaring which parts are ****, doing a dozen forks that aren't compatible with each-other, fixing one device and breaking another, the whole circus. I'd rather it's at least semi-stable before everybody goes running off with it. And I don't need everybody looking over my shoulder or explaining/debating every line of code thrice while re-doing half the core.

Patience... also because now its no longer weekend and I have contract work to do aside from this.

Also, if one were to incorporate this into a custom firmware, I would use a very different methods than I did in Pry-Fi. I might actually be committing those changes to Omni, but I can't do everything at once.
 
Last edited:

banderos101

Senior Member
Dec 26, 2010
1,261
374
FYI, I feel releasing source at this point is premature. I'm not really trying to hide anything here. If you really want the source right now I'll even give it to you.

However, a lot of devices work in a slightly different way, and I wanted to get it out there to see how it would work out. After a lot of feedback, test versions, releases, etc (even though its only been 3 days) I feel a part of the core logic may need to be redesigned to incorporate the new knowledge (some even from this thread) to make it work better.

What I specifically don't want is a dozen or so source happy OSS zealots jumping on it, declaring which parts are ****, doing a dozen forks that aren't compatible with each-other, fixing one device and breaking another, the whole circus. I'd rather it's at least semi-stable before everybody goes running off with it. And I don't need everybody looking over my shoulder or explaining/debating every line of code thrice while re-doing half the core.

Patience... also because now its no longer weekend and I have contract work to do aside from this.

Also, if one were to incorporate this into a custom firmware, I would use a very different methods than I did in Pry-Fi. I might actually be committing those changes to Omni, but I can't do everything at once.

i think the main concern with many folks, as you just pointed out, is the idea of a root app and no way to review code, but thats not to pick on anyone in particular, its anyone and everyone, even cadilacs amongst men such as youself chainfire:) ....... its just a general thinking, sometimes folks like myself will sacrafice that concern if app/author usability outweighs that concern, so something like open source would alleviate that somewhat.......

im glad to hear the reason and your thoughts on this, happy to see where you are hoping to take it
 

kenshin33

Senior Member
Mar 25, 2009
333
133
Montreal
If you could quickly re-hash why there is no extra security in disabling SSID broadcasts, without having to review the entire Vivek video series, that'd be great.

http://www.howtogeek.com/howto/2865...hiding-your-wireless-ssid-really-more-secure/

Have you actually reviewed this (and solutions like this) ? Do they even turn Wi-Fi fully off, including the background scanning ? AFAIK that's not possible without leveraging some form of root or system modification. And if it does uses that, it should be listed as a con because its a "security" app that runs as root. If it's a con for one app...
I use LLAMA to disable wifi when not home or at the university b/c of battery and also I don;t really trust wifi hotspots out there, if there's a malicious one out there you PNL and mac address are the least of your concernes (especially the "OK/CONTINUE" trigger happy users).
GPS locks fairly quicly most of the time, if not I enable wifi at that point just to get a fix quicker, (the only time I really need it is for bus schedules, -- any other extended --navigation-- use will need very high acuracy and GPS should be on, so no need to drain the battry more with wifi), but that's just me.


As for a solution, finding a way to enable passive scan mode would be great, and sould solve the leak, no root not fiddeling needed (passive mode don;t send beacons, just listen to beacons sent out by AP, and for the life of me I don;t understand why would a client sned a beacon while the AP is expected to?)

Random : see what cm did with tuna kernel (random but not so random)

just my 2 cents

Also there's ap_scan variable in wpa_supplicant, not sure what it does but it have some influance.
 
Last edited:

cernekee

Senior Member
Jun 2, 2013
186
427
you can may be copy it ?

I could copy the code that populates the MAC address, and omit the parts that calculate it from the OMAP chip ID. But that doesn't tell me where my unique N7 WLAN-firmware-supplied MAC address actually came from.

I'll have to do some more research here...

I didn't see you mention this anywhere else, but the above doesn't appear to be entirely true, or is incomplete. On my test devices, there aren't any networks at all with scan_ssid=1, yet my MBA running Wireshark 24/7 (finally a use for a Mac) regularly intercepts the list of SSIDs (if Pry-Fi is not enabled). Certainly not as often as normal scans, but they're definitely still broadcast.

Are you seeing any of these messages in logcat: "Scan SSID", "Starting AP scan for specific SSID:", or "Include wildcard SSID in the scan request"?

Are there real AP names in the request, or just P2P names?

The code in wpa_supplicant_scan() which writes out the list of SSIDs to pass to the driver via params.ssids is:

Code:
		while (ssid) {
			if (!wpas_network_disabled(wpa_s, ssid) [b]&&[/b]
			    [b]ssid->scan_ssid[/b]) {
				wpa_hexdump_ascii(MSG_DEBUG, "Scan SSID",
						  ssid->ssid, ssid->ssid_len);
				params.ssids[params.num_ssids].ssid =
					ssid->ssid;
				params.ssids[params.num_ssids].ssid_len =
					ssid->ssid_len;
				params.num_ssids++;
				if (params.num_ssids + 1 >= max_ssids)
					break;
			}
			ssid = ssid->next;
			if (ssid == start)
				break;
			if (ssid == NULL && max_ssids > 1 &&
			    start != wpa_s->conf->ssid)
				ssid = wpa_s->conf->ssid;
		}

In my tests, the logic did work as expected. Some factors that may account for variations could include:

  • Android version and/or wpa_supplicant version: 4.2.2 and wpa_supplicant_8 v2.0-devel-4.2.2
  • OS: CM/AOSP, no Samsung
  • Drivers: bcmdhd in the kernel, and nl80211 in wpa_supplicant
  • Observation time (not 24/7 here), and on-demand scans vs. scheduled scans

If you could quickly re-hash why there is no extra security in disabling SSID broadcasts, without having to review the entire Vivek video series, that'd be great.

While disabling SSID broadcasts takes the string out of the beacon, it still appears in the association requests. So there are two easy ways to expose the SSID of a hidden network:

  • Run airodump-ng, or the equivalent point-and-click GUI tool, and just passively wait around for a client to (re)associate. Then the name will pop up on the screen.
  • Use "aireplay-ng --deauth" to kick one STA (unicast) or all STAs (broadcast) off the network. Most will reassociate, leaking the SSID name.

Also, from what I've seen, hiding the SSID doesn't consistently turn off the beacons; it just sends an empty SSID field. So the AP is still broadcasting its presence + BSSID.

As for strange interactions, as documented, this is caused by the app having to change Wi-Fi states to let the MAC changes work. If it works as (currently) intended all you'd see is Wi-Fi going off/on again rapidly once or twice, and only if you are watching the settings->wifi screen. If anything else weird is going on, feel free to elaborate.

I ran into the "forget network" caveat earlier. It would be nice to make this more seamless.

Also, if one were to incorporate this into a custom firmware, I would use a very different methods than I did in Pry-Fi.

Agreed - that is one reason I started a separate thread to discuss options.

Personally I don't mind hacking my ROM if it results in a cleaner, but less generic, solution.

It's not a security app, it's a privacy app, and you can't do this on a non-custom firmware without root, so I don't really see the point about complaining that its a security app that runs as root - most apps that do anything significant security or privacy wise do

banderos101 pretty much covered this. No issue with running as root, as long as it's easy to see (and change) how it all works.

Security- and privacy- conscious individuals tend to get a little control-freaky about what they're running on their systems:

I don't need everybody looking over my shoulder or explaining/debating every line of code thrice

In security/privacy/crypto circles, that is usually a feature not a bug. ;)
 

kenshin33

Senior Member
Mar 25, 2009
333
133
Montreal
I could copy the code that populates the MAC address, and omit the parts that calculate it from the OMAP chip ID. But that doesn't tell me where my unique N7 WLAN-firmware-supplied MAC address actually came from.

I'll have to do some more research here...
The function as it is just populates the mac in a buffer, if it has to fail it will fail later , while "setting it", Unfortunatly I can't test it out (my grouper has a broken screen, flo/hammerhead should, so no point in testing).
 

Chainfire

Moderator Emeritus / Senior Recognized Developer
Oct 2, 2007
11,452
87,856
www.chainfire.eu
Are you seeing any of these messages in logcat: "Scan SSID", "Starting AP scan for specific SSID:", or "Include wildcard SSID in the scan request"?

I'll see if I can correlate the timings of the logcat messages with the scans

In my tests, the logic did work as expected. Some factors that may account for variations could include:

  • Android version and/or wpa_supplicant version: 4.2.2 and wpa_supplicant_8 v2.0-devel-4.2.2
  • OS: CM/AOSP, no Samsung
  • Drivers: bcmdhd in the kernel, and nl80211 in wpa_supplicant
  • Observation time (not 24/7 here), and on-demand scans vs. scheduled scans

FWIW, I've done the main testing on a stock+root Nexus 5 @ 4.4.2. This is the one broadcasting all the SSIDs. I think I recall seeing it on my Samsung test devices as well, but I'd have to test again to be 100%.

While disabling SSID broadcasts takes the string out of the beacon, it still appears in the association requests. So there are two easy ways to expose the SSID of a hidden network:

  • Run airodump-ng, or the equivalent point-and-click GUI tool, and just passively wait around for a client to (re)associate. Then the name will pop up on the screen.
  • Use "aireplay-ng --deauth" to kick one STA (unicast) or all STAs (broadcast) off the network. Most will reassociate, leaking the SSID name.

Ah this is a misscommunication then. I thought we were talking about the added security of not broadcasting every SSID your device has ever heard of, not of the practise of making an AP hidden.

I ran into the "forget network" caveat earlier. It would be nice to make this more seamless.

Hence the manage networks option inside the app (which still also doesn't work half the time) ... This definitely needs some work.

Agreed - that is one reason I started a separate thread to discuss options.

Personally I don't mind hacking my ROM if it results in a cleaner, but less generic, solution.

Agreed. That's not my own primary aim though. The stock+root crowd is factors larger than the custom ROM crowd, so they come first to me.

For Pry-Fi itself I think a large chunk of weirdness can be covered by moving a part of code to use wpa_cli commands instead. But first I have to test how well that even works cross-OEM. That alone will take a few days, even before implementing anything. So many devices - and at least Samsung modifies wpa_supplicant.

banderos101 pretty much covered this. No issue with running as root, as long as it's easy to see (and change) how it all works.

Security- and privacy- conscious individuals tend to get a little control-freaky about what they're running on their systems:

I know, but passive aggressive statements and generally being more naggish than a pack of wild Jehovah's Witnesses got tiring a few decades ago - I can't possibly reward that behavior.

In security/privacy/crypto circles, that is usually a feature not a bug. ;)

You are undoubtedly used to working with a different class of coders than I am. If I had a dollar for every pages-long criticism based on not actually reading the code, or vague assumptions made without actually reading the code, or trivial questions that could be answered simply by reading the code, or in-depth discussions about code clearly marked as 'should be replaced by something less half-assed' ... it just ruins my productivity as well as all will to code. After a chunk of code is complete is a different matter, of course, I just don't like being annoyed during my code time.

You can say it sucks when the painting is finished, not before.
 
Last edited:

banderos101

Senior Member
Dec 26, 2010
1,261
374
I know, but passive aggressive statements and generally being more naggish than a pack of wild Jehovah's Witnesses got tiring a few decades ago - I can't possibly reward that behavior.


You can say it sucks when the painting is finished, not before.

Ah, maaan, but im so good at passive aggresive statements, wheres the love :(

Joking aside, i see your point, to be honest im surprised your still with us, but glad you still are, otherwise, no pry-fi, and many other great apps, and contributions

so, sorry, and i hate you :angel::mad:.........passive aggressive, got it licked

Disclaimer:
i dont hate you
 
Last edited:

Chainfire

Moderator Emeritus / Senior Recognized Developer
Oct 2, 2007
11,452
87,856
www.chainfire.eu
I'll see if I can correlate the timings of the logcat messages with the scans

I'm not sure the reason why, but it seems the wpa_cli I compiled from AOSP does does not play nice with the stock wpa_supplicant on my Nexus 5 (potentially killing a lot of Pry-Fi improvements).

Bot claim to be "v2.1-devel-4.4.2" though.

I can't get any of the commands you have listed to do anything useful. For example, "scan" just throws "unknown command" at me.

All I really see coming out at this time is "IFNAME=wlan0 <3>CTRL-EVENT-SCAN-RESULTS"

EDIT My bad, I had the wrong paramters for wpa_cli, re-checking now

EDIT#2 "Starting AP scan for specific SSID:" happens just before the full-list broadcast, but before a broadcast without the full list of SSID names as well. I don't really see any difference between the scans based on logcat.

EDIT#3 A full SSID-listing scan appears to happen at 120 second intervals
 
Last edited:

kenshin33

Senior Member
Mar 25, 2009
333
133
Montreal
I'm not sure the reason why, but it seems the wpa_cli I compiled from AOSP does does not play nice with the stock wpa_supplicant on my Nexus 5 (potentially killing a lot of Pry-Fi improvements).

Bot claim to be "v2.1-devel-4.4.2" though.

I can't get any of the commands you have listed to do anything useful. For example, "scan" just throws "unknown command" at me.

All I really see coming out at this time is "IFNAME=wlan0 <3>CTRL-EVENT-SCAN-RESULTS"

EDIT My bad, I had the wrong paramters for wpa_cli, re-checking now

EDIT#2 "Starting AP scan for specific SSID:" happens just before the full-list broadcast, but before a broadcast without the full list of SSID names as well. I don't really see any difference between the scans based on logcat.

EDIT#3 A full SSID-listing scan appears to happen at 120 second intervals

I'm trying the same thing, how did you go arround the UNKNOWN COMMAND?


line 748 of scan.c (wpa_supplicant)
somehow max_ssid is set to 1. Why is the question? (ap_scan == 2 might be the answer, but it is not set in wpa_Supplicant.conf AFAI can tell)
The comment seem to indicate that this is done out of convinience (wpa_supplicant can be rebuilt to avoit this behaviour)
 

Top Liked Posts

  • There are no posts matching your filters.
  • 11
    Over the past year or so, a number of news stories have discussed the possibility of tracking smartphone users in public via wifi probes. The recent announcement of Pry-Fi inspired me to do a little research to see exactly what information was being revealed, and what could be done about it.

    The easiest way to observe the information leak in question (and later, to find out if it has been successfully plugged) is to follow part 4 of Vivek Ramachandran's WLAN Security Megaprimer. He describes the process of setting up a virtual machine running Backtrack Linux (now replaced by Kali Linux), and using aircrack-ng and wireshark to observe WLAN activity.

    The whole series is worth watching, but here are a few basic survival commands:

    Code:
    # enable RF monitor mode on wlan0 (your interface might have a different name)
    airmon-ng start wlan0
    
    # compile a list of known networks and their BSSIDs; hit ^C to quit
    airodump-ng mon0
    
    # tune the WLAN card to the channel of interest (seen in the above list)
    iwconfig wlan0 channel 1
    
    # start wireshark and then begin sniffing
    wireshark

    Android uses the standard wpa_supplicant infrastructure found on Linux PCs to manage the WLAN interface and its database of known access points (APs). The configuration is stored in /data/misc/wifi/wpa_supplicant.conf. I was not able to find this mirrored anywhere else (e.g. in a SQLite database).

    The wpa_supplicant daemon is controlled through commands sent over its standard socket interface by $AOSP/frameworks/base/wifi/java/android/net/wifi/WifiInfo.java, its JNI helpers, and libhardware. If your ROM supports it, you can observe what wpa_supplicant is doing by running wpa_cli from a shell, then in another window, running 'adb logcat -s "wpa_supplicant:*"'. Some useful wpa_cli commands include:

    Code:
    # show each outbound message (other levels include: info, debug, excessive)
    log_level msgdump
    
    # initiate a scan and dump results (note that the results also show up in
    # logcat if the log_level is sufficiently high)
    scan
    scan_results
    
    # show current status
    status
    
    # show known networks
    list_networks
    
    # disable scan_ssid on entry #2 from list_networks
    set_network 2 scan_ssid 0
    save
    
    # reread the wpa_supplicant.conf file
    reconfigure

    On my N5 running KK 4.4 I needed to preface most commands with "IFNAME=wlan0", but on my N7 running JB 4.2.2 only the bare commands were accepted. YMMV so try both if in doubt.

    Here are some observations gathered from experimenting with Nexus 7 (2012) running JB 4.2.2:

    • There is an entry in wpa_supplicant.conf for every known network.
    • Any network added manually with the '+' icon will have scan_ssid=1. Any network picked from the list of scan results will not have a scan_ssid property.
    • When wifi is enabled and the user is in the Settings->Wifi activity, Android will perform a network scan about every 8 seconds.
    • When wifi is enabled and the user is doing something else, and the interface is not currently associated with an AP, Android will perform a network scan about every 15 seconds.
    • On each scan, wpa_supplicant will send out probe requests. Every probe request contains the interface's MAC address. :(
    • wpa_supplicant will send probe requests containing an explicit ESSID name for each entry that has scan_ssid=1. If there are no such entries, the SSID parameter in the probe request will always be empty (signifying a broadcast probe request).
    • If wpa_supplicant.conf contains a very common ESSID name (like "linksys" or "netgear") then your device will try to associate with any other AP that uses that name. i.e. as expected, it matches the ASCII ESSID rather than the BSSID MAC address
    • If your device associates with a network, it leaks a whole bunch more data (and could be victimized by a malicious network). So if privacy/security are important, you'll want to carefully control the conditions under which your device associates with wifi networks.
    • N.B. The baseband side might not be any more trustworthy, but that is beyond the scope of this discussion.

    The latter two items potentially leak sensitive data: your MAC address uniquely identifies your smartphone, and an ESSID list can reveal networks to which you have connected in the past (home networks, work networks, etc). In some cases the network names may be unique enough to translate back to a specific geographical location. The ESSID is just an ASCII string, not the AP's MAC address, so it may introduce some degree of ambiguity.

    If you're monitoring wpa_supplicant via wpa_cli and logcat, and you used "log_level msgdump", you can see which ESSID names your device is broadcasting:

    Code:
    D/wpa_supplicant(19967): wlan0: Control interface command 'SCAN'
    D/wpa_supplicant(19967): wlan0: Setting scan request: 0 sec 0 usec
    [color=red]D/wpa_supplicant(19967): Scan SSID - hexdump(len=7): 61 74 74 77 69 66 69[/color]
    D/wpa_supplicant(19967): wlan0: Starting AP scan for wildcard SSID
    D/wpa_supplicant(19967): WPS: Building WPS IE for Probe Request
    D/wpa_supplicant(19967): WPS:  * Version (hardcoded 0x10)
    D/wpa_supplicant(19967): WPS:  * Request Type
    D/wpa_supplicant(19967): WPS:  * Config Methods (4388)
    D/wpa_supplicant(19967): WPS:  * UUID-E
    D/wpa_supplicant(19967): WPS:  * Primary Device Type
    D/wpa_supplicant(19967): WPS:  * RF Bands (1)
    D/wpa_supplicant(19967): WPS:  * Association State
    D/wpa_supplicant(19967): WPS:  * Configuration Error (0)
    D/wpa_supplicant(19967): WPS:  * Device Password ID (0)
    D/wpa_supplicant(19967): WPS:  * Manufacturer
    D/wpa_supplicant(19967): WPS:  * Model Name
    D/wpa_supplicant(19967): WPS:  * Model Number
    D/wpa_supplicant(19967): WPS:  * Device Name
    D/wpa_supplicant(19967): WPS:  * Version2 (0x20)
    D/wpa_supplicant(19967): P2P: * P2P IE header
    D/wpa_supplicant(19967): P2P: * Capability dev=24 group=00
    D/wpa_supplicant(19967): P2P: * Listen Channel: Regulatory Class 81 Channel 6
    [color=red]D/wpa_supplicant(19967): nl80211: Scan SSID - hexdump(len=7): 61 74 74 77 69 66 69[/color]
    [color=blue]D/wpa_supplicant(19967): nl80211: Scan SSID - hexdump(len=0): [NULL][/color]

    The red lines show probe requests that leak the "attwifi" ESSID and your MAC address. The blue line shows a probe request with an empty (broadcast) ESSID, which still leaks your MAC address.

    So, what can we do about these leaks? I'll post a list of suggestions and see what else the XDA community comes up with:

    • Check your wpa_supplicant.conf for any entries with scan_ssid=1. If the network has SSID broadcasts disabled, you need this property; if you are the administrator you can re-enable SSID broadcasts on the AP and then eliminate scan_ssid=1. Vivek's videos explain why disabling SSID broadcast on the AP does not actually provide any security benefits.
    • Disable "Keep Wi-Fi on during sleep." This option is found under Settings -> Wi-Fi -> (menu) -> Advanced. If wifi is disabled while the phone is in your pocket sleeping, it will not broadcast its MAC address every 15 seconds. This isn't a comprehensive solution but it reduces the scope of the problem.
    • Run Smarter Wi-Fi Manager. SWM uses your coarse (cellular/GPS?) location to decide whether or not to enable the WLAN radio. Pros: open source; eliminates many possible information leaks. Cons: I believe this requires location services enabled, which may supply location data to other apps and/or impact battery life.
    • Run Pry-Fi. Pros: I have verified through Wireshark that this indeed randomizes the MAC addresses in the Probe Request packets. Cons: Not open source (and it's a "security" app that runs as root); causes strange interactions with the standard Android UI. Other potential side effects are unknown as the code has not yet been analyzed.
    • Modify the ROM. It may be possible to tweak wpa_supplicant or the kernel WLAN driver to send random or generic (ff:ff:ff:ff:ff:ff?) MAC addresses in their probe requests. For wpa_supplicant this may be doable through Cydia Substrate. It is likely that a well-done ROM modification would have few or no side effects on usability. However, it may require each individual wpa_supplicant driver or kernel WLAN driver to be modified.
    • Modify the UI. It would be nice if the Settings -> Wi-Fi interface clearly distinguished entries with scan_ssid=1. Maybe these could be shown in red with a simple Xposed module or ROM tweak.

    Any other ideas?
    7
    FYI, I feel releasing source at this point is premature. I'm not really trying to hide anything here. If you really want the source right now I'll even give it to you (cernekee)

    However, a lot of devices work in a slightly different way, and I wanted to get it out there to see how it would work out. After a lot of feedback, test versions, releases, etc (even though its only been 3 days) I feel a part of the core logic may need to be redesigned to incorporate the new knowledge (some even from this thread) to make it work better.

    What I specifically don't want is a dozen or so source happy OSS zealots jumping on it, declaring which parts are ****, doing a dozen forks that aren't compatible with each-other, fixing one device and breaking another, the whole circus. I'd rather it's at least semi-stable before everybody goes running off with it. And I don't need everybody looking over my shoulder or explaining/debating every line of code thrice while re-doing half the core.

    Patience... also because now its no longer weekend and I have contract work to do aside from this.

    Also, if one were to incorporate this into a custom firmware, I would use a very different methods than I did in Pry-Fi. I might actually be committing those changes to Omni, but I can't do everything at once.
    2
    • When wifi is enabled and the user is in the Settings->Wifi activity, Android will perform a network scan about every 8 seconds.
    • When wifi is enabled and the user is doing something else, and the interface is not currently associated with an AP, Android will perform a network scan about every 15 seconds.

    The background scan intervals are determined by these security settings (http://androidxref.com/4.4.2_r1/xref/frameworks/base/core/java/android/provider/Settings.java):

    wifi_supplicant_scan_interval_ms
    wifi_framework_scan_interval_ms

    which in turn are sourced from these config values:

    config_wifi_supplicant_scan_interval (15000 in AOSP)
    config_wifi_framework_scan_interval (300000 in AOSP)

    Inside the settings activity, it is determined by the WIFI_RESCAN_INTERVAL_MS field which is 10000 by default in AOSP (http://androidxref.com/4.4.2_r1/xre...ifi/WifiSettings.java#WIFI_RESCAN_INTERVAL_MS)

    • wpa_supplicant will send probe requests containing an explicit ESSID name for each entry that has scan_ssid=1. If there are no such entries, the SSID parameter in the probe request will always be empty (signifying a broadcast probe request).

    I didn't see you mention this anywhere else, but the above doesn't appear to be entirely true, or is incomplete. On my test devices, there aren't any networks at all with scan_ssid=1, yet my MBA running Wireshark 24/7 (finally a use for a Mac) regularly intercepts the list of SSIDs (if Pry-Fi is not enabled). Certainly not as often as normal scans, but they're definitely still broadcast.

    • Check your wpa_supplicant.conf for any entries with scan_ssid=1. If the network has SSID broadcasts disabled, you need this property; if you are the administrator you can re-enable SSID broadcasts on the AP and then eliminate scan_ssid=1. Vivek's videos explain why disabling SSID broadcast does not actually provide any security benefits.

    If you could quickly re-hash why there is no extra security in disabling SSID broadcasts, without having to review the entire Vivek video series, that'd be great.

    • Run Pry-Fi. Pros: I have verified through Wireshark that this indeed randomizes the MAC addresses in the Probe Request packets. Cons: Not open source (and it's a "security" app that runs as root); causes strange interactions with the standard Android UI. Other potential side effects are unknown as the code has not yet been analyzed.

    It's not a security app, it's a privacy app, and you can't do this on a non-custom firmware without root, so I don't really see the point about complaining that its a security app that runs as root - most apps that do anything significant security or privacy wise do (requiring a framework/system modification and then not requesting root access is not an excuse, you still need root access for this at some point).

    As for strange interactions, as documented, this is caused by the app having to change Wi-Fi states to let the MAC changes work. If it works as (currently) intended all you'd see is Wi-Fi going off/on again rapidly once or twice, and only if you are watching the settings->wifi screen. If anything else weird is going on, feel free to elaborate.

    • Run Smarter Wi-Fi Manager. SWM uses your coarse (cellular/GPS?) location to decide whether or not to enable the WLAN radio. Pros: open source; eliminates many possible information leaks. Cons: I believe this requires location services enabled, which may supply location data to other apps and/or impact battery life.

    Have you actually reviewed this (and solutions like this) ? Do they even turn Wi-Fi fully off, including the background scanning ? AFAIK that's not possible without leveraging some form of root or system modification. And if it does uses that, it should be listed as a con because its a "security" app that runs as root. If it's a con for one app...

    Also I would like to list as an additional con that this (fairly dramatically) reduces location determination speed and accuracy for location-based apps, which may or may not be an issue for the user.
    1
    Has anyone done any testing for possible side-effects when using MAC address randomization while using VPN, large downloads or other persistent connection?
    1
    Has anyone done any testing for possible side-effects when using MAC address randomization while using VPN, large downloads or other persistent connection?

    Provided the address doesn't change between sessions (only on association and connection) there is no issue with changing addresses.