[HOWTO][exFAT][WORK IN PROGRESS] Mount exFAT formatted drives and cards

I have successful compiled the exFAT userspace driver from http://code.google.com/p/exfat/ together with libfuse.
So we are theoretically able to mount every exFAT formatted drive (connected via OTG and also the external sdcard, BUT THIS IS NOT YET FULL TESTED).
This howto is far away from being perfect. Also my english isn't perfect - sorry. Feel free to send me corrections.

@Mods: I think it's a development-related thread. If this is not your mind, please move it to the right section - thank you very much (and also for your continuously work in the "background").

* For updates please have a look in the footer of this post, I forgot to reserve a second "post space" *

Please be very careful! I am not responsible for any damage or lost data on your phone or storage. I have tested this on my phone with a connected card reader and with the "external" sdcard
(Galaxy S2, usually mounted at /sdcard/external_sd).

ATTENTION:
I have discovered one "big" problem that must be solved before all other things and I NEED PERHAPS SOME HELP!
Binaries attached read update no. 2:

After every reboot the exFAT partition will be damaged WITHOUT modification of the vold.fstab config, so don't use a card or drive with important data on it.
This has to do with the automount function of the "New Volume Manager Daemon" Vold.
I suspect that the daemon wants to mount the exFAT volume as FAT32 read/write and overwrites the first bytes of the the first block. I will check this.
It doesn't matter if it's the "external" card or a connected drive/sdcard reader.

The problem:
dd if=/dev/block/mmcblk1p1 of=/sdcard/exfat_ok.bin count=1 bs=1024
hexdump -C /sdcard/exfat_ok.bin

the correct hex:
00000000h: EB 76 90 45 58 46 41 54 20 20 20 (three bytes and then the string EXFAT plus three spaces. This is the recognition string for the exfat-utils (exactly 8 bytes)

dd if=/dev/block/mmcblk1p1 of=/sdcard/exfat_not_ok.bin count=1 bs=1024
hexdump -C /sdcard/exfat_not_ok.bin

after a reboot of the phone the string changes to
00000000h: 52 52 61 41 58 46 41 54 20 20 20 (four new bytes at the beginning -> AXFAT, the recognition of the partition fails) I think no more changes are made.

This is not in relation to the exFAT tools or the FUSE library, the "damage" happens before!

Beside that, the exFAT card seems to be usable and after manually mount to /sdcard/external_sd the card can be activated (under settings - memory) - Sorry I have a german GUI...

STOP READING HERE IF YOU DON'T WANT TO PLAY WITH YOUR DATA ON THE EXFAT PARTITION!

There is no long term experience with this. Don't try it if you don't have some skills with Linux and Android. Make a full backup!
You have been warned...

[Q] Why exFAT and not using ext2/3/4 or any other file system?
[A] I don't know. It's your decision. exFAT is developed mainly for flash memory and could be used "out of the box" with newer windows versions and Mac OS X in contrast to ext2/3/4.
Sadly there is no good extX windows driver out available.
exFAT also supports XDHC card from 32 GB upwards. See http://en.wikipedia.org/wiki/ExFAT for more details.
XDHC cards are reported to work with the SG2.
This makes it interesting for micro sdcards greater than 32 GB (if your are lucky and have one) and for large files (greater than 4 GB), like video files.
Note: FAT32 is of course able to address more than 32 GB, but not "official".
There are many disadvantages as well, read the article.

[Q] Is it free and/or free to use?
[A] Once again, I don't know exactly. Tuxera http://www.tuxera.com/ has an agreement (licence program) with Microsoft and announced an exFAT driver for Android, but I couldn't find a free downloadable (source) package.
The driver is probably free to use but not free to distribute and until now not released.

The exFAT project on Google Project Hosting is licensed under GNU GPL v3, so we will and can use it free. But it's still in beta stage.

[Q] What do I need?
[A] A rooted Android phone with a suitable kernel and fuse support, take siyah (because it a good kernel). I have only a Samsung Galaxy S2, that's because the thread is here.
Enough free memory. Installed busybox. Access via adb shell or a ssh-terminal connection (QuickSSHd or SSHDroid from the market).
An other option is to use a terminal window on the phone.

[Q] What could be possible?
[A]

• automatically mount the OTG drive with util-linux-ng or vold or something like that (needs support from kernel or ROM)
• replace the FAT32 partition of the "external" sdcard (external_sd) with exFAT (needs support from both kernel and ROM I think)

OK, let's start...

First option, the harder way (you need a linux machine for this): Compiling the driver and utils

1. Download the CodeSourcery Toolchain/Crosscompiler for ARM EABI for Linux from https://sourcery.mentor.com/sgpp/lite/arm/portal/subscription?@template=lite (tested with 2011.03-41, I saw a newer untested version Sourcery G++ Lite 2011.03-42)
2. Install the compiler on your linux box with (for example) sh ./arm-2011.03-41-arm-none-linux-gnueabi.bin, the installer will asking you a few questions, it should be easy.
3. Download latest stable fuse (fuse-2.8.6.tar.gz) from http://fuse.sourceforge.net/
4. Download fuse-exfat (exfat-utils-0.9.5.tar.gz and fuse-exfat-0.9.5.tar.gz) from http://code.google.com/p/exfat/ or use my prepared packages with the Makefiles
5. Prepare the cross compiler, this is my example script, please change the path (CROSS_PATH) to the CodeSourcery binaries and the CROSS_ROOT path
   
   
   Make a directory (CROSS_ROOT) for the libraries and the headers, ex.:
   /android/src/cross/lib
   /android/src/cross/include
   
   ---- File prepare_codesourcery.sh
   #!/bin/bash
   export CROSS_PATH=/android/CodeSourcery
   export CROSS_ROOT=/android/src/cross
   
   export ARCH=arm
   export PATH="$CROSS_PATH:$PATH"
   
   # version 2011.03-41-arm-none-linux-gnueabi
   export CROSS_COMPILE="$CROSS_PATH/bin/arm-none-linux-gnueabi-"
   export CFLAGS=' -I$CROSS_ROOT/include -g -O2 -static -march=armv6 -mfpu=neon -mfloat-abi=softfp'
   export LDFLAGS=' -L$CROSS_PATH/arm-none-linux-gnueabi/libc/lib -L$CROSS_ROOT/lib -Wl,--whole-archive -lpthread -lrt -ldl -Wl,--no-whole-archive'
   export CC="$CROSS_PATH/bin/arm-none-linux-gnueabi-gcc"
   ----
   
   
6. Export the setup:
   . ./prepare_codesourcery.sh
   
   Check the path with
   arm-none-linux-gnueabi-gcc -v
   
   
   
7. Compile libfuse
   • ./configure --host=arm-linux --enable-util --enable-lib --disable-mtab --enable-static=yes --enable-shared=no
   • make
   • copy the static libraries libfuse.a and libulockmgr.a from fuse-2.8.6/lib/.libs to $CROSS_ROOT/lib
   • copy the headers (.h-files) from fuse-2.8.6/include to $CROSS_ROOT/include
   c) and d) is not really necessary, but is used for the flags to find the headers and libs, see the file prepare_codesourcery.sh above.
   
   
   
8. Compile fuse-exfat
   I had no luck with SCons (a substitution for make) to cross compile for ARM, so I created some Makefiles to build fuse-exfat and exfat-utils, see attachement
   
   Note: the next step is not necessary, libexfat is also included in exfat-utils if you use the attached source package:
   
   Use your downloaded sources package and copy the Makefiles from my packages to every directory or use my source packages
   In fuse-exfat/fuse-exfat-0.9.5 run make
   
   
   
9. Compile exfat-utils
   In fuse-exfat/exfat run make
   
   
   Note: If you get errors like strip: "Unable to recognise the format of the input file" then you have to symlink arm strip to strip temporary with
   ln -s $CROSS_PATH/bin/arm-none-linux-gnueabi-strip $CROSS_PATH/bin/strip
   so arm-strip is used instead of strip from your linux dist
   
   
   
   Second option: Download the binaries
   
   
   1.-9. Don't care about it...
   
   
10. Copy all binaries to your phone. They are big but "portable" because of the static build. You can use adb or any other method. The files must be executable, so place them for example in /system/xbin or /data/ and chmod them 755
    
11. Connect an empty hard drive, empty pen drive or a card reader with an empty sdcard to the phone with an OTG cable. The drive should only contain a prepared partition (don't care about the file system).
    But you can also create a partition with fdisk on the phone, if your busybox installation is useable.
    
    • Check the connection of the USB devices with
      lsusb
      or something like that
      
    • Check the partitions with
      cat /proc/partitions
      You have to see a new partition like sdc1. The partition is visible under /dev/block/platform/s3c_otghcd/sdc1
      
    • Create a new exFAT partition with
      mkexfatfs /dev/block/platform/s3c_otghcd/sdc1
      Check the type of partition with
      fdisk -l /dev/block/platform/s3c_otghcd/sdc
      (you should see it as "HPFS/NTFS")
      
    • Make a new directory ex.
      mkdir /data/exfat
      for the mount point
      
    • Mount the new exFAT partition read/write with
      mount.exfat-fuse -o rw /dev/block/platform/s3c_otghcd/sdc1 /data/exfat
      to mount point /data/exfat or any other path
      or for testing with
      mount.exfat-fuse -o ro /dev/block/platform/s3c_otghcd/sdc1 /data/exfat
      readonly
      
    • To unmount the device use
      sync
      umount /data/exfat

Thanks to gokhanmoral for his great kernel, tolis626 and olifee (members of this forum) to give me the idea of doing this, unknown devs from http://repository.timesys.com/ for a example Makefile to bypass
the unwieldy "SCons". I wasn't able to use it for cross compiling because of tons of parameters and variables, my shame...


Links:
exFAT (GPL): http://code.google.com/p/exfat/wiki/QuckStartGuide (it's not a typo)
Some informations: http://en.wikipedia.org/wiki/ExFAT
SiyahKernel: http://xdaforums.com/showthread.php?t=1263838
exFAT Makefile: http://repository.timesys.com/buildsources/f/fuse-exfat/fuse-exfat-0.9.5/fuse-exfat-0.9.5-make.patch
CodeSourcery: https://sourcery.mentor.com/sgpp/lite/arm/portal/subscription?@template=lite
FUSE (Filesystem in Userspace): http://fuse.sourceforge.net/


Update no. 1 | 01/12/2011
I think I made one step forward: It's vold as I can see.
I have commented out the block for the external_sd in /system/etc/vold.fstab
# external sdcard
#{
# ums_path = /sys/devices/platform/usb_mass_storage/lun1/file
# asec = enable
#}
#dev_mount sdcard1 /mnt/sdcard/external_sd auto /devices/platform/s3c-sdhci.2/mmc_host/mmc1

Now after new rebooting the partition is not damaged and I was able to mount it as expected at /sdcard/external_sd.
The disadvantage is now is that the system cannot recognize the card as a regular sdcard and the memory part in settings is greyed out.
It's like the card is not insert for the ROM, nevertheless the directories are shown in a file explorer like "root explorer" (with free/used values and I could edit a text file with a build-in editor)

Update no. 2 | 05/12/2011
I am now sure after some (more) tests, it's the vold daemon.
To mount a exfat volume, the configuration /system/etc/vold.fstab needs modification.

DO NOT MOUNT A VOLUME WITHOUT MODIFICATION

vold (version 2) is locked to VFAT/FAT32 volumes. Earlier versions had support for ext(2/3/4 ???) volumes too, this was removed by Google and/or Samsung (don't know).

Sadly I can't find a documentation for vold2 and I am stuck here. Because for replacing the "external_sd" from FAT32 to exFAT it's also necessary that vold2 recognizes the card correctly. (Because of the "asec" mounts for Apps2SD).
Perhaps it's possible to map this mounts to the internal sdcard (setting asec = enable in vold.fstab), but I haven't tried this yet.

Conclusion: It's possible to mount such exFAT volume with some restrictions and with modification of the vold.fstab.



Specs:
/data/bin/dumpexfat /dev/block/mmcblk1p1
dumpexfat 0.9.5
Volume label
Volume serial number 0xb965fe93
FS version 1.0
Sector size 512
Cluster size 32768
Sectors count 25173456
Free sectors 25169728
Clusters count 393284
Free clusters 393277
First sector 0
FAT first sector 128
FAT sectors count 3136
First cluster sector 3264
Root directory cluster 5
Volume state 0x0000
FATs count 1
Drive number 0x80
Allocated space 0%

Please no questions about the values, there is a second ext4 partition on the card...

So the configuration in vold.fstab and perhaps some other files have to be changed. I have nearly no knowledge with "void". Is a expert out there?
From command line a short speed test shows this result (no other GUI test possible in the moment):

/data/bin/hdparm -tT /dev/block/mmcblk1p1

/dev/block/mmcblk1p1:
Timing cached reads: 228 MB in 2.01 seconds = 113.27 MB/sec
Timing buffered disk reads: 36 MB in 3.02 seconds = 11.92 MB/sec

Card: Patriot 16 GB Class 10, no OC

RESERVED

Reserved...

z3r0n3 said:

Did you test the overhead of a FUSE filesystem on Android? As far I know the performance may be sub-optimal because of the overhead of using a filesystem on userspace mode. The Tuxera driver uses kernel mode and is very optimized, but as far I know it's only for OEM's that want to license the module for their devices.

Anyway, very interesting, mainly because exFAT is the default filesystem for SDXC.

Click to expand...

Click to collapse

No, I have not tested this. It will be one of the next steps if there is a solution for the problem I wrote about. In the moment I don't know if it's kernel related, rom/vold related or anything other. But I think it should have less overhead than NTFS in userspace, surely more than FAT32. Please read the comments about speed at http://code.google.com/p/exfat/updates/list And yes, the Tuxera driver is not for us "end users". Perhaps Samsung will give us a present in the next official ROM release

dildano said:

First of all, thanks for your work on this. I was shocked to discover that my SGS2 wouldn't support any filesystems that support large files on external sdcards. I was able to get your solution working in the sense that I could format an sdcard with exfat, mount it, write to it, and unmount it. Awesome! I am having one serious problem though: After editing the vold.fstab and rebooting, I get constant FCs after trying to install any APKs. It doesn't matter if I have my external_sd mounted or not. And these are not apps that are trying to install to the sdcard. Any ideas?

I've got the AT&T version of the SGS2 (i777), but I'm running Siyah's latest kernel. The FCs just say it's the media process.

Click to expand...

Click to collapse

Sorry for my later answer!

This should be nevertheless a problem with the app2sd service and I have no solution for this. My thread here is only a howto, nothing for the "daily to use"...
I also don't know why there is no other selectable alternative file system for "us users" to use for the external card by default and not FAT32.

The media service depends on the vold daemon to my knowledge. So I cannot recomment this for the default external sdcard, because of the media service scans. If you want to store larger files (e.g. video files), you should better split your sdcard in one FAT32 partition and an additional ext2/3/4 partition. On this partition there is no 4 GB limit. The "media scanner" will not scan the files (videos etc.) on the partition, but you can choose the videos from your favorite player with the file chooser.
Hope this helps you a little bit.

dildano said:

Thanks for the response. I was actually trying out multiple partitions over the weekend to no avail. Granted, I was trying a combination of FAT32 and NTFS. It would actually work for a while until the NTFS partition would appear to get corrupted. I thought about ext2, but my understanding is that Samsung somehow disabled ext* support for external SD cards. Is that not the case? Anyway, I'm surprised that more folks haven't caught onto your work here because storing large files seems to be a fairly common issue for SGS2 users.

Click to expand...

Click to collapse

I am using an ext3 partition with deactivated journalling and noatime option on the sdcard since months without problems. Of course I have also a FAT32 partition on the card to stay compatible with vold and media scanner. With a start script it is mounted after every phone restart.

shadowofdarkness said:

I would like to try this, for use connecting my camera which has a exfat formatted sdxc card in it via OTG. But have a couple questions since I can't test at this moment.

1: Do the binaries work on a ICS build (Siyah Kernel) or do they need updated?
2: Do I have to make any vold.fstab changes to prevent FS damage when using a OTG cable or is that just external sd?
3: If I have to make the changes will other devices (not memory) work normal without manual interaction?

Click to expand...

Click to collapse

Hello, I am still on GB, but I guess it should work on ICS too.
If ICS (or better the vold daemon) has not changed its behavior you have to modify vold.fstab to prevent damages. To do this, you have to disable automounting the exfat OTG device (commenting out the part for OTG and reboot). The other partitions (internal and external memory should work like before). Then mount your camera card manually. But be very careful with your data/pictures!

WhiteTrap said:

Hello, I'm working on an HTC Mazaa with Windows Phone and inside a partition I've found several times the EXFAT header, so I'm trying to mount it, but looks like it's EXFAT 2.0, while your implementation covers only 1.0.

Do you know if it's somehow possible to mount EXFAT 2.0? On Windows, on Linux, modifying your tool, whatever!

Here's the header:

Code:

012f7400  eb 76 90 45 58 46 41 54  20 20 20 00 00 00 00 00  |.v.EXFAT   .....|
012f7410  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
012f7440  00 00 00 00 00 00 00 00  00 1b 00 00 00 00 00 00  |................|
012f7450  20 00 00 00 36 00 00 00  8c 00 00 00 74 1a 00 00  | ...6.......t...|
012f7460  02 00 00 00 94 01 eb 07  00 02 10 00 09 00 02 80  |................|
012f7470  ff 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
012f7480  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
012f75f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
012f7600  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
012f77f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
012f7800

Thanks!

Click to expand...

Click to collapse

I don't think that I can give you a good answer, sorry. It's not my tool, it's the exFAT driver from http://code.google.com/p/exfat/, the latest is fuse-exfat-0.9.7.tar.gz and I don't see any notes for a "version 2" in the ChangeLog. Are you really sure about the version? If there is a version 2, I guess it's more than only a change to the header.
I recommend you to post the question in the mailing list https://groups.google.com/group/exfat?hl=en. It is rumored that exFAT works natively on the new S3, but I don't know if this will be true and I have strong doubts that Samsung will release the sources of the driver.

shadowofdarkness said:

I just wanted to say thanks I just tested this and the binaries works great on ICS 4.0.3 (LPG) using Siyah Kernel.

I plugged in my Panasonic Lumix TS2 digital camera with a 64GB sdxc card and was able to mount it fine on my S II with a OTG cable.

Click to expand...

Click to collapse

Fine and thanks for sharing your experience! It's good to hear that the driver is useful for you.