[HOWTO][exFAT][CM10.1] Working 64GB ext sdcard with fuse-exfat (1.0.1) + FIX broken

[HMkX2]

This post has largely been relegated obsolete by my Flashable Mod.
Please see my new post >>HERE<<

I am leaving this up for informational purposes, and for record-keeping.​

[HOWTO][exFAT][CM10.1] Working 64GB ext sdcard with fuse-exfat (1.0.1) + FIX broken

How to get a exFat formatted 64GB external sdcard working with CM10.1 using fuse-exfat & FIX broken card headers

Attribution credits:

Original thanks go to originator of the method, smitna in the Galaxy S2 forum, for providing the method. Additional thanks go to shardul_seth (Post 23) for compiling the latest version of the fuse-exfat binaries for ARM, and of course Andrew Nayenko for his efforts in creating GPL'd exFat code (fuse-exfat on google code).[HOWTO][exFAT][WORK IN PROGRESS] Mount exFAT formatted drives and cards​

TL;DR - READ THE POST FIRST!:

1. Download shardul_seth's binaries for fuse-exfat from (Post 23) or the bottom of this post.
2. Unzip the files into /data/ (requires root)
3. Open an ADB shell as root (requires USB-Debugging turned on, ADB utility, also at bottom)
4. Run the following commands: (change mmcblk1p1 to suit your sdcard's partitioning - it might be mmcblk1. DO NOT FSCK WITH blk0, that is your internal card!!!)
   

   chmod 775 /data/*fat*
   mount -o rw,remount /system
   echo "/data/mount.exfat-fuse -o rw,dirsync,umask=0 /dev/block/mmcblk1p1 /storage/sdcard1" > /etc/init.d/10exfatsdmount
   chown root:shell /etc/init.d/10exfatsdmount
   chmod 755 /etc/init.d/10exfatsdmount

   Done! Reboot and enjoy.
   
   SHOULD YOU BORK YOUR CARD BY REINSERTING IT (or get "Card Damaged" icon)
   
   
5. Run the included filesystem header patcher script as root (warning: has minimal safety checks, is hardcoded for "mmcblk1p1") to fix the likely corruption: (easily modified)
   

   /data/fixmyexfat.sh

6. Reboot. (or run "vold"?)

Proof / What to Expect:

Overview:

An open source project recently reverse-engineered the exFat specification via a fuse driver (ala fuse-ntfs-3g). The first stable reelase was in January, and I am astounded that it hasn't been implemented in CM yet -- aside from the moral and potential legal reasons. A user here at XDA compiled the binaries and posted instructions, I am taking on the lead for putting it all together for others. I am tired of seeing posts dated as late as February 11th saying it is "Impossible" and to give up, or flash another ROM -- you are spreading misinformation and it needs to be stopped. This implementation isn't fully vetted, but it works seamlessly enough for most users.

Even without the fuse driver, it's pretty well known CM10.1 (and CM10) mangle exFat cards -- and to fix them, you need to merely run "chkdsk /f" on the drive from Windows. However, as soon as you plug it back into the phone, it automounts and mangles it again before you can do anything. I found out what was getting mangled, why, and when, and tried to work around it. I found by mounting at boot-time, before vold has a chance to attempt an automount, you can get the system to fully recognize a card. (See screenshots.) Theoretically, this means you can have a SDcard with multiple partitions -- a "dummy" fat32 partition at the start to fool vold with the mount scripts, and the exfat one that you rebind over it. (System -> Storage dynamically updates as long as you don't "unmount" from the GUI.)

The hex dump on the left is the first 512 bytes of a healthy exFat partition, the second from a mangled one. By noting the offsets and exact mangling of (0x000 = "RRaA", 0x1E4 = "rrAa", 0x1FC = 00 00 55 AA), I found out that it was attempting to replicate the "FS Information Sector" Signature of FAT32 - so there is a problem with CM's code somewhere attempting a fsck.fat32 when it isn't appropriate.

CM 10.1 / Android 4.2.1 also completely redid how mountpoints are set up. There are summarized in the "Technical Details" section below. The gist of it is, /data/block/vold/179:96 is the device (e.g. /sda/) and /data/block/vold/179:97 is the first partition (/sda1/). And in CM10.1 (if you had a FAT32 card), the 97 one is mounted to /storage/sdcard1/ via FUSE. However, these block-special files are dynamically generated by vold so are inappropriate to use -- we need to use mmcblk1p1. Depending on how the card is formatted, your mounts may change slightly.

The final challenge is where to put the binaries. I could not chmod make files on the /storage/sdcard0/ mount executable for some reason - by looking at CM10.1's source code I found why. Android moved to using a "fake" vfat partition and MTP under a FUSE layer unify storage space and remove file corruption/mutex issues. Unfortunately, that means it blocks permissions --- hence my recommendation to put the binaries in /data/ which is exposed as ext4. If you absolutely must put the binaries on the internal SDcard, you need to run the chmod commands from /data/media/0/ folder and the permissions will mirror properly. (This is the ext4 original of /storage/sdcard0/ "vfake".)


Technical Details: (mounts, softlink chains)

[B]ALL-IMPORTANT "FIXER" HEX-PATCHING CODE[/B]
(Reminder: dd = Destroyer of Disks. I take no responsibility.)

echo -e "\xeb\x76\x90\x45"|dd of=/dev/block/mmcblk1p1 conv=notrunc bs=1 count=4 seek=0
echo -e "\xff\xff\xff\xff"|dd of=/dev/block/mmcblk1p1 conv=notrunc bs=1 count=4 seek=484
echo -e "\x1f\x2c\x55\xaa"|dd of=/dev/block/mmcblk1p1 conv=notrunc bs=1 count=4 seek=508

[B]FOR INTERNAL SD CARD, "USER #0"[/B]

/storage/[B]sdcard0[/B] -> /storage/emulated/legacy
/storage/emulated/[B]legacy[/B] -> [B]/mnt/shell/emulated/0[/B] (fuse-fake-fat!)

/data/media/0 (ext4)

/dev/fuse [B]/mnt/shell/emulated fuse[/B] rw,nosuid,nodev,relatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0

[B]FOR FAT32 EXTERNAL SD CARD[/B]

/dev/block/vold/179:96 /storage/[B]sdcard1[/B] vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0

(Notice it is [I]179:96[/I] -- the whole disk is storage. If you had it on Partition1, it would be [I]179:97[/I])

[B]FOR exFAT EXTERNAL SD CARD[/B]

/dev/block/mmcblk1p1 /storage/sdcard1 fuseblk rw,dirsync,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,blksize=4096 0 0

(In this case, it is [I]mmcblk1[B]p1[/B][/I] -- since I partitioned it. However, I have fudged my card before so it was just  [I]mmcblk1[/I].)

If you need/want to change between "Floppy" mode (single big disk) and "Hard Disk" mode (with a partition table), I highly recommend using a program called BOOTICE to do it. It supports a number of "quirks" like reordering visibility of NTFS partitions on a Flash Drive, etc. (I will add instructions if there is demand.)

Remaining issues:

Unmounting poses no problems, doesn't corrupt anything. However, upon remounting (eg: vold rescan, physically removing/reinserting the card -> autodetect, or clicking "Mount SD Card" in the GUI), vold will re-mangle the filesystem header. The notification of "Damaged SD Card" also gets displayed and cannot be removed, even if my hotfix script is applied live.

There was a version of Vold-exfat floating around for ICS, but that likely isn't compatible for us.​

MTP works. UMS however is not currently implemented in the Unofficial Mastamoon build of CM10.1 I use. I therefore cannot test how Windows handles it.

Programs like XDA's own "SGS3 Easy UMS" do not work, or work behaves eratically. By a convoluted scheme (Unmounting card from Shell -> Plugging in usb -> clicking "UMS") I was able to see a Read-only CD-Rom drive of my SD card. This seems to be a common issue.
Manual methods like changing the build.prop do not work - you see an *empty* CD-rom drive.​

(From what I can gather) Cyanogen 10.1 does not currently allow for plug-and-play fuse modules to allow you to pick your own filesystem just by adding a .ko. Similarly, the exfat-utils (exfatfsck, etc) fail to run (even on a "fixed" card) since DKIOCGETBLOCKSIZE in io.c cannot detect the block size -- you will notice "blkid" doesn't recognize the exfat device either. "Vold" gets confused and mangles the filesystem header.

Cyanogenmod needs to be updated to support Fuse modules more gracefully -- and ABSOLUTELY NOT modify the filesystem at all unless prompted.​

 

[DonHuan]

Sorry, problem solved.

 

[HMkX2]

Flashable mod is up, created new thread in the right section. Check OP for link.