[GUIDE] [INFO] All about SD CARDS - which one, why and how! Posts 1-3.

Questions on external storage - SD cards - seem to go on and on and on...

There is a large amount of confusion about them and the information (and mis-information) that exists is all over the web.

Which one is the fastest? The best? How do I partition and format them? Or, the best for my device?

There is no simple answer since the performance of these cards depends on many factors, not of the least of these being the card itself.






If you don't like to read you are likely in the wrong thread.. Anyway, if you just want my bottom line - here it is..




Counterfeit card info in post 2.
Formatting info in post 3.



COMMONLY USED CARD TYPES

The most commonly used card type for mobile phones is the SD (Secure Digital) card. It includes 4 families available in different form factors. The four families are the original Standard-Capacity (SDSC), the High-Capacity (SDHC), the eXtended-Capacity (SDXC), and the SDIO, which combines input/output functions with data storage. The three form factors are the original size, the "mini" size, and the "micro" size. There are many combinations of form factors and device families.


DEFINING BASIC CARD SPEED

SD memory card manufacturers use different types of flash memory to create cards, so actual transfer speeds can vary. Varying speeds make it difficult to determine which card will provide reliable recording of streaming content (which is what most manufacturers seem to be concerned with these days).

The two primary classifications are Speed Class and UHS Speed Class.

"Speed Class" has been used by many as a guide to their performance. The Class of a card is a general indication of the read/write speed. Unfortunately things are not that simple in real life. A card's speed depends on many factors, a few of which are:

The likelihood of soft errors that the card's controller must re-try. Writing data requires the controller to read and erase a larger region, then rewrite that entire region with the desired part changed.​

Buffers used by the operating system which, in combination with the card's format, sector size and even the device's card bus speed can impact performance.​

The possibility of fragmentation - that a body of information the host views as a unit is written to non-contiguous regions of memory. This does not cause rotational or head-movement delays as with magnetic media, but it does vary the amount of computation the card's controller must do.​

Speed Classes 2, 4, and 6 assert that the card supports the respective number of MB/s as a minimum sustained write speed for a card in a fragmented state. Class 10 asserts that the card supports 10 MB/s as a minimum non-fragmented sequential write speed.

The following speed classes are defined:

Class 2 - 2 MB/sec

Class 4- 4 MB/sec

Class 6- 6 MB/sec

Class 10 - 10 MB/sec

UHS-I (UHS class 1) - a theoretical 50 MB/sec

USH-II - a theoretical 312 MB/sec

The Ultra-High Speed (UHS) designation is available on some SDHC and SDXC cards. UHS rated cards support a clock frequency of 100 MHz (a quadrupling of the original "Default Speed"), which in four-bit transfer mode could transfer 50 MB/s or a clock frequency of 208 MHz, which could transfer 104 MB/s. Double data rate operation at 50 MHz (DDR50) is mandatory for microSDHC and microSDXC cards labeled as UHS-I. In this mode, four bits are transferred when the clock signal rises and another four bits when it falls, transferring an entire byte on each full clock cycle. UHS-II cards further raise the data transfer rate to a theoretical maximum of 312 MB/s.

It should be noted that it is quite possible for a good Class 4 card to outperform a not so good Class 10 card in various devices (including the HTC HD2 which is what I am using at the time of this writing) as a result of various factors, including card quality, buffers, read/write errors, operating system and application software methodology used.

So, in theory Class 10 is faster than Class 4. As an example, recording video requires a constant minimum write speed while random access (such as our phones mostly use) does not rely on the same type of access. This is really great if you are often recording long videos but I suspect most of us use our phones a little differently.

Often you can get higher speeds from a card than it is rated for. For example, my Class 4 32 GB SanDisk card reads much faster than one might expect it and all too often it is the read speed that makes the device "snappy" (like loading an app).


RANDOM TESTS

It should be noted that these tests were not done by myself. There are many places where tests are described and many methodologies used for them. Benchmarking card performance is not a simple task.

Some of the cards tested include Sandisk 16G class 4, Memorystar 8 G class 10, Memorystar 16 G Class 10, Kingston 8 G class 10, Kingston 16 G class 10, Lexar HS Mobile 32 G class 10, Lexar HS Mobile 16 G class 10, Patriot LX 16 G Class 10, Adata 8 G class 6, Samsung plus 8 G class 6, Sandisk 32G class 4.

Random reads of 512 KB blocks show the SanDisk Mobile Ultra microSDHC to be the fastest card at 21.8 MB/s.

Reducing the block size for the random read test to 4 KB significantly impacts performance, which tops out at a mere 3.4 MB/s, achieved by the Samsung and Adata Class 6 cards. There is almost no difference between queue depths of one and 32 when it comes to memory cards.

Click to expand...

Click to collapse

Click to expand...

Click to collapse

You can review the entire test and the results here.

The random tests and results, which impact mobile phones more, are located here.


BENCHMARKING YOUR CARD

Benchmarks are a rather elusive science since there are too many variables that can impact the results, not the least of which is the card itself. Anyone who has worked with SD cards in the real world has probably noticed that the brand of card can make a dramatic difference in performance. Tests have shown as much as a 12x difference in performance between best and worst cards in real world tasks. As already noted, the "class" ratings used by manufacturers as a measure of performance are not reliable indicators of performance either. In fact, some manufacturer's class 2 and class 4 rated cards routinely outperform cards rated as class 6 or class 10!

Consistency of testing is important since different devices and bottlenecks, such as bus speeds, buffers, file systems and activity can all impact and alter the results. This is why a card that has performed extremely well on a test may turn out to be a dog in your device. Short of testing all cards in a lab environment under exacting circumstances may well yield inconsistent results.

There are a lot of apps available for benchmarking, for Android, as well as Windows and Linux. The following short list is not a recommendation of any software.

HD Tune is a hard disk utility with many functions. It can be used to measure the drive's performance, scan for errors, check the health status (S.M.A.R.T.), securely erase all data and much more. It was designed for hard disks and it has not been updated in recent times. While it could be used for SD Cards (and has been) it is not the ideal tool for it.

More info is available here.​

AndroBench is a popular and up to date benchmark application that measures the storage performance of Android devices. It includes Micro benchmarks, SQLite benchmarks and Macro-benchmarks (see screenshot below).

More info is available here.​

Crystal Disk Mark is another hard disk test utility which is also somewhat dated (last revision dates back to 2010 as far as I know). It runs under various versions of Windows and Internet Explorer.

More info is available here.​

AnTuTu Benchmark is Android Benchmarking tool for Android devices. It can run a full test, through the "Memory Performance","CPU Integer Performance","CPU Floating point Performance","2D 3D Graphics Performance","SD card reading/writing speed","Database IO" performance", etc. The many tests it can perform and the frequent updates make it a viable tool for benchmarking.

More info is available here.​

Vellamo is a reasonably easy-to-use suite of system-level benchmarks for devices based on Android 2.3 and later. Some of the functions it offers include CPU subsystem performance, scrolling and zooming, 3D graphics, video performance, memory read/write and peak bandwidth performance.

[ EDIT ] The app has recently been updated and is available on Google Play and is now compatible with KitKat.

More info is available here.​

SD Tools is probably one of the most often used tools to check microSD cards (Name, Date, MID, OEMID). You can check if your card is fake. (Check serial number and MID and OEMID). You can also benchmark sd card writing and reading speeds. It is fast and easy to use. See screenshot below.

More info is available here.​

Here's my 32GB Sandisk Class 4 card on the Android speed test:

And here is the same card as seen through Androbench:

FILE SYSTEMS FOR SD CARDS

The traditional view has been that storage is not a huge factor for performance on mobile devices. Flash storage (the type most commonly used today) draws little power, and its performance is thought to exceed that of the network subsystem. Yet, oddly enough, just by varying the flash storage, performance over WiFi can typically vary between 100% to 300% across applications; in one extreme scenario the variation jumped to over 2000%. The relationship between storage and application performance seems to be a combination of flash device performance, random I/O from application databases, and use of synchronous writes. Changes to the storage subsystem can significantly improve user experience.

The three primary (not sole) factors impacting device performance as it relates to storage seem to be the media (the card itself), the file system used and the quality of the applications used.

Speed Class is largely irrelevant as it is not necessarily indicative of application performance; although the class rating is meant for sequential performance, there are several cases in which higher-class SD cards performed worse than lower-class ones overall. If not addressed, lower performing storage not only makes the application run slower, it also increases the energy consumption of the device. This part is intended to deal with file systems used in mobile devices and will not address media or application quality (sorry, no tutorial on proper programming techniques) in great detail although they both have a significant impact on performance.



Briefly, Android (as it pertains to storage) consists of flash storage, operating system and Java middleware, and applications; the OS itself is based on Linux and contains low-level drivers (e.g., flash memory, network, and power management), Dalvik virtual machine for application isolation and memory management, several libraries (e.g., SQLite, libc), and an application framework for development of new applications using system services and hardware.

The Dalvik VM is a fast register-based VM providing a small memory footprint; each application runs as its own process, with its own instance of the Dalvik VM. Android also supports true multitasking and several applications usually run as background processes; processes continue running in the background when you leave an application (e.g., a browser downloading web pages).

Android uses SQLite database as the primary means for storage of structured data. SQLite is a transactional database engine that is lightweight, occupying a small amount of storage and memory; it is thus popular on embedded and mobile operating systems. Applications are provided a well defined interface to create, query, and manage their databases; one or more SQLite databases are stored per application on the data partition.

The YAFFS2 file system, managing raw NAND flash was traditionally the file system of choice for the various internal partitions including /system and /data; it is lightweight and optimized for flash storage. Recently, Android transitioned to Ext4 as the default file system for these partitions. Several other files systems have been implemented with varying success in Android devices.

More than 50 file systems have been documented for Linux alone and attempting to document all of them here is simply not possible for me. So many file systems, such little time! You can find plenty of information on all them by doing only a few internet searches. I will concentrate on relatively few of them here. Note that not all attributes of a given file system may be implemented in a given Android system. The omission of a file system from this post should not reflect negatively on it, but rather on my time, or lack thereof.

YAFFS (Yet Another Flash File System) was designed and written by Charles Manning. It is a log-structured file system that holds data integrity as a high priority. Yaffs1 works on NAND chips that have 512 byte pages + 16 byte spare areas. Newer NAND flash chips have larger pages (2048 bytes + 64 bytes spare areas) and stricter write requirements. Each page within an erase block (128 kilobytes) must be written to in sequential order, and each page must be written only once. YAFFS2 was designed to accommodate these newer chips.

YAFFS has been used on Linux, WinCE, pSOS, eCos, ThreadX, and various special-purpose OSes. YAFFS initialization simply erases flash memory. When a bad block is encountered, it follows the smart media scheme of marking the fifth byte of the block's spare area. Blocks marked as such remain unallocated from then on. To write file data, YAFFS initially writes a whole page (chunk in YAFFS terminology) that describes the file metadata, such as timestamps, name, path, etc. The new file is assigned a unique object ID number; every data chunk within the file will contain this unique object ID within the spare area. YAFFS maintains a tree structure in RAM memory of the physical location of these chunks. When a chunk is no longer valid (the file is deleted, or parts of the file are overwritten), YAFFS marks a particular byte in the spare area of the chunk as ‘dirty’. When an entire block (32 pages) is marked as dirty, YAFFS can erase the block and reclaim the space. When the filesystem's free space is low, YAFFS consolidates a group of good pages onto a new block. YAFFS then reclaims the space used by dirty pages within each of the original blocks.

When a YAFFS system mounts a NAND flash device, it must visit each block to check for valid data by scanning its spare area. With this information it then reconstitutes the memory-resident tree data structure.​

The extended file system, or ext, was implemented in 1992 as the first file system created specifically for the Linux kernel. It has metadata structure inspired by the traditional Unix File System and was designed by Rémy Card. It was the first implementation that used the virtual file system and it could handle file systems up to 2 gigabytes in size.

The ext2, ext3 and ext4 file systems were all derived from this one. Most ext discussions center around ext3 and ext4 in the Android world.

ext3 is a journaled file system that is commonly used by the Linux kernel. Its main advantage over ext2 is journaling, which improves reliability and eliminates the need to check the file system after an unclean shutdown. Generally, ext3 is slower than competing Linux filesystems, such as ext4, JFS, ReiserFS and XFS, but it has a significant advantage in that it allows in-place upgrades from ext2 without having to back up and restore data. Benchmarks suggest that ext3 also uses less CPU power than ReiserFS and XFS. It is also considered safer than the other Linux file systems, due to its relative simplicity and wider testing base. ext3 does not do checksumming when writing to the journal and if the hardware is doing out-of-order write caching, you run the risk of severe filesystem corruption during a crash.

ext4 was created as a series of backward compatible extensions to ext3. In January 2010, Google announced that it would upgrade its storage infrastructure from ext2 to ext4. In December 2010, they also announced they would use ext4, instead of YAFFS, on Android. The ext4 advantages include large file system support, extents, persistent pre-allocation and journal checksumming.​

NILFS (New Implementation of a Log-structured File System) is a log-structured file system for Linux. It is being developed by Nippon Telegraph and Telephone Corporation (NTT) CyberSpace Laboratories. It uses a copy-on-write technique known as "nothing in life is free", NILFS records all data in a continuous log-like format that is only appended to, never overwritten, a design intended to reduce seek times, as well as minimize the kind of data loss that occurs after a crash with conventional file systems. For example, data loss occurs on ext3 file systems when the system crashes during a write operation. When the system reboots, the journal notes that the write did not complete, and any partial data writes are lost. NILFS also includes fast write and recovery times, minimal damage to file data and system consistency on hardware failure, 32-bit checksums, etc.

Android kernels do not routinely include NILFS although mods to make it available can be found.​

F2FS (Flash-Friendly File System) was created by Kim Jaegeuk at Samsung for the Linux operating system kernel. The motivation for it was to build a file system that from the start takes into account the characteristics of NAND flash memory-based storage devices, which have been widely used in computer systems ranging from mobile devices to servers. Samsung chose a log-structured file system approach, which it adapted to newer forms of storage. F2FS also remedies some known issues of the older log structured file systems, such as the snowball effect of wandering trees and high cleaning overhead. Because a NAND-based storage device shows different characteristics according to its internal geometry or flash memory management scheme (such as the Flash Translation Layer), Samsung also added various parameters not only for configuring on-disk layout, but also for selecting allocation and cleaning algorithms. Introduced in the second half of 2012 this new file system shows promise but is not yet generally available in any Kernels that I have seen. Samsung has submitted these patches for integration into the Linux kernel, which means it’s likely to appear on Android releases in the future.​

JFFS2Journalling Flash File System version 2 or JFFS2 is a log-structured file system for use with flash memory devices. It is the successor to JFFS. JFFS2 has been included in the Linux kernel since the 2.4.10 (2001-09-23) release. JFFS2 is also available for a few bootloaders, like Das U-Boot, Open Firmware, the eCos RTOS and the RedBoot. Most prominently JFFS2 is used in OpenWrt. At least three file systems have been developed as JFFS2 replacements: LogFS, UBIFS, and YAFFS.

JFFS2 adds support for NAND flash devices which have a sequential I/O interface and cannot be memory-mapped for reading. It does not include hard links (this was not possible in JFFS because of limitations in the on-disk format) but it does have compression. Four algorithms are available: zlib, rubin, rtime, and lzo. It claims better performance - JFFS treats the disk as a purely circular log which generats a great deal of unnecessary I/O. The garbage collection algorithm in JFFS2 makes this mostly unnecessary.​

Today's predominant file system, YAFFS2, will likely be replaced in the future by the likes of ext4, nilfs or f2fs. In order to do a fair comparison one must compare I/O throughput, user data access latency, application execution latency and data safety. Not a simple task.

If you ever want to get Linux techies arguing just talk about which file systems are the best.

Google, which knows a thing or two about fast systems, has decided, for their purposes anyway, that Ext4 is the best and close to the fastest file system of all. Google also hired Ted T'so, who also happens to be the leading Ext4 programmer. In a note to the Ext4 developer mailing list, Google's Michael Rubin, a senior staff engineer, wrote, "Google is currently in the middle of upgrading from ext2 to a more up to date file system. We ended up choosing ext4." So, if you are using an Android phone and you are not a kernel developer you may want to take Google's word for it, at least for now, and go with the ext4 file system on your SD Card.

In all fairness, the numerous tests that have been ran over the years will prove different winners. Some show ext2 to slightly outperform ext4 but we must also consider data safety and journaling. While not many of us will have vital, enterprise data on our mobile devices, reconfiguring and restoring a device can be tedious, at best. Some will argue for NILFS, others for ext4, and yet others.. well, you get the idea.


I cannot tell you which is the best or fastest or safest. What I can tell you is that, based on my experience, I am staying mostly with ext4 for the time being - reasonable speed and safety combination for my needs.



Comments? Additions? Suggestions? They are all welcome.
Flame wars (relating to SD Cards or otherwise) are not. :-]

Formatting and Troubleshooting

PARTITIONING AND FORMATTING

The many file systems available for these cards is beyond the scope of this post but a brief mention is needed, nonetheless. Likewise, deciding on read buffers and cluster sizes depend a great deal on your operating system, how the card is used, the speed of the card bus and other factors. You will have to read and experiment more if you want to get optimum performance from your card.

Because the host views the SD card as a block storage device, the card does not require MBR partitions or any specific file system. The card can be reformatted to use any file system the operating system supports such as UFS, Ext2, Ext3, Ext4, btrfs, HFS Plus, HFS Plus, NTFS, FAT16, FAT32, exFAT, etc.

Most consumer products that take an SD card will expect it to be partitioned and formatted in some way. The universal support for FAT16 and FAT32 allow the usage of SDSC and SDHC cards on most host devices. On such SD cards, standard utility programs can be used to repair a corrupted filing system and sometimes recover deleted files. Defragmentation tools for FAT file systems may be used on such cards but are generally not recommended. The resulting consolidation of files may provide a marginal improvement in the time required to read or write the file, but not an improvement comparable to defragmentation of hard drives, where storing a file in multiple fragments may involve a time penalty to move between physical areas of the drive. Moreover, defragmentation performs writes to the SD card that count against the card's rated lifespan.

An SD card should have a life span of roughly 10 years (which doesn't mean it always will). In theory, the more often you write/erase it the faster it will wear. Of course, replacing an SD card is fairly inexpensive these days.

The memory of a card is divided into minimum memory units. The device writes data onto memory units where no data is already stored. As available memory becomes divided into smaller units through normal use, this leads to an increase in non-linear, or fragmented storage. The amount of fragmentation can reduce write speeds, so faster SD memory card speed standards help compensate for fragmentation.

Reformatting an SD card with a different file system, or even with the same one, may make the card slower, or shorten its lifespan. Some cards use wear leveling, in which frequently modified blocks are mapped to different portions of memory at different times, and some wear-leveling algorithms are designed for the access patterns typical of the file allocation table on a FAT16 or FAT32 device. In addition, the preformatted file system may use a cluster size that matches the erase region of the physical memory on the card; reformatting may change the cluster size and make writes less efficient.

The SD Association provides free formatting software to overcome many problems described above. This software formats all SD memory cards, SDHC memory cards and SDXC memory cards. It was created specifically for memory cards using the SD/SDHC/SDXC standards. It is generally recommended to use the SD Formatter first if the card is to be reformatted by another method. Using generic formatting utilities may result in less than optimal performance for your memory cards.

Nota bene: cards greater than 32GB will be automatically formatted as exFat by this software! If you require another file system, such as FAT32, you will have to do a second format with another utility! If you need to format a 64 GB card to FAT32 you can use various utilities to get it done, including this FAT32Format program.

The SD/SDHC/SDXC memory cards have a "Protected Area" on the card for the SD standard's security function. The SD Formatter does not format the "Protected Area". This is genrally reserved for the use of the device, such as cameras and mobile phones.

To create other partitions, such as an "ext4" Linux formatted partition, grab a copy of MiniTool Partition Wizard. The Home Edition is a free partition manager software designed by MiniTool Solution Ltd. It supports 32/64 bit Windows Operating Systems. Functions include: resizing partitions, copying partitions, create partition, extend partition, split partition, delete partition, format partition, convert partition, explore partition, etc.

Similar to the MiniTool Partition Wizard is the Paragon Partition Manager which also has a free version you can download. This one will allow you to increase an ext4 partition's cluster size, if you wish to.

This is a very good article on partition alignment if you are interested in knowing more about that subject - thanks MarkAtHome!

​

The free SD Association formatting software is available here.

The free Minitool Partition Manager software is available here.

The free Paragon Partition Manager software is available here.




There are many ways you can format a card, including various recovery tools, some bootloaders, PC card readers, etc. I am not suggesting that other methods do not work or that they are bad. This is simply my preferred way of getting it done.


Let's take a 32 GB SanDisk card and set it up for some of the devices. I am using a card reader attached to my PC for these steps. If this is not a brand new card the first step I will take is format it with SD Formatter to make sure it is in optimal form. Loading the program on your PC will give you a screen like this one:

This program will create the "proper" format for your card. Note that this means a 64 GB card which was originally sold with the eXFat format will be so formatted. If your device requires FAT32 format (which many phones require) you will have to do that with another program.

Note the "Option" button! Clicking this button will let you select the format type (quick, full or erase). I generally select full although it takes quite a bit longer. I am seldom concerned with securing old data from other people. I do, however, like to have format adjustment turned on.

When SD Formatter is finished you should see something like this:

If you need only a FAT32 partition you can stop here. You are done. If your setup requires a Linux type partition also for apps or data or whatnot you keep going.

​

My ROM uses the extended partition for most everything except the Android system. The advantage of this separation is that reads and writes will deal with two different devices, hopefully making things a little faster than waiting for just one.

I prefer to use the ext4 file format which includes journaling (see writeup on file system for more info). I will accomplish the creating of this partition with the Minitool Partition Manager. When you first load the program you should see something like this image although the number of drives and their order will depend on your system.

Note that the FAT32 partition is marked as "Primary"! This was done automatically by SD Formatter but if you are using another program you must remember to do this yourself or chances are the partition will not been properly seen on your device!

Creating the second, ext4 type partition also requires it to be set as "Primary"!

WARNING!! This program requires you click the apply button in order for your selections to be completed. If you do everything and forget to click apply nothing will actually be done!!

​

As you can see, the FAT32 partition spans the entire card. The simplest way to create room for another partition is to first decrease the size of the FAT32 partition. You can select the "Partition" menu choice on top or right click the FAT32 partition and you will see a "Move/Resize" choice. Selecting it will bring you to this window:

Reduce the size of the FAT32 partition and you will see available space increase as you are doing it. I am aiming for a 3 GB ext4 partition:

Once you apply the changes you should see the unallocated space available for the second partition:

Right click the newly created space and select "Create" partition. Don't forget to set the new partition as "Primary"! Since I am not planning to create another partition (such as Linux swap) I am using all the free space on the card.

Select Apply again and you should see your second partition being created:

Once the process is completed both of your partitions should be clearly shown and, if you did it right, both will be flagged as "Primary" partitions! Now, that wasn't so difficult, was it?

A FEW WORDS ABOUT PARTITION TYPES

There is apparently a lot of information floating around on the topic of "partition types". While they are fairly well defined and documented I could find nothing specific about requiring FAT32 type "b" partitions for Android. That does not mean I could not have missed the info but one would think it should be quite easy to find if this was a blanket requirement.

I read through Tytung's git repositories (okay, kinda speed reading since there is way too much source code there for limited time reading) and I could not find any restrictions on FAT32 type, though again, admittedly I could have missed it.

As importantly, I created a FAT32 partition as type "b" on one card and another as type "c" on another card - they both worked equally well for me. Minitool Partition Wizard creates type "c" (the FAT32 default for the program) unless you change it yourself. I have apparently never used anything other than type "c" on my HD2, on Typhoon GB ROMs, and Tytung's ICS and JB ROMs. So, you got me. If it matters I have not seen that yet.

Briefly, the partition type (or partition ID) in a partition's entry in the partition table inside a Master Boot Record (MBR) is a byte value intended to specify the file system the partition contains and/or to flag special access methods used to access these partitions, such as CHS (Cylinder/Head/Sector) mappings, LBA (Logical Block Addressing) access, logical mapped geometries, special driver access, hidden partitions, secured or encrypted filesystems, etc.

It is up to an operating system's boot loader and/or kernel how to interpret the value.

Type "b" was the original (or older) WIN95 OSR2 FAT32 design. It is limited to partition sizes of 2047GB or less and it depended on the BIOS INT 13 of those PCs. Type "c" also dates back to WIN95 OSR2 FAT32 but is LBA-mapped and is considered an extended-INT 13 equivalent of 0b. Of course, both of these specs were designed for actual (hard) disk drives. According to some notes I have seen the "c" revision should be faster but there is no concrete evidence of this as far as I have seen.

By changing the partition type ID, users can prevent the system from using or initializing partitions - this is up to the operating system which will be accessing it.

According to Microsoft the following limitations exist using the FAT32 file system with Windows operating systems:

• Clusters cannot be 64 kilobytes (KB) or larger. If clusters were 64 KB or larger, some programs (such as Setup programs) might calculate disk space incorrectly.
• A volume must contain at least 65,527 clusters to use the FAT32 file system. You cannot increase the cluster size on a volume using the FAT32 file system so that it ends up with less than 65,527 clusters.
• The maximum possible number of clusters on a volume using the FAT32 file system is 268,435,445. With a maximum of 32 KB per cluster with space for the file allocation table (FAT), this equates to a maximum disk size of approximately 8 terabytes (TB).
• The ScanDisk tool included with Microsoft Windows 95 and Microsoft Windows 98 is a 16-bit program. Such programs have a single memory block maximum allocation size of 16 MB less 64 KB. Therefore, The Windows 95 or Windows 98 ScanDisk tool cannot process volumes using the FAT32 file system that have a FAT larger than 16 MB less 64 KB in size. A FAT entry on a volume using the FAT32 file system uses 4 bytes, so ScanDisk cannot process the FAT on a volume using the FAT32 file system that defines more than 4,177,920 clusters (including the two reserved clusters). Including the FATs themselves, this works out, at the maximum of 32 KB per cluster, to a volume size of 127.53 gigabytes (GB).
• You cannot decrease the cluster size on a volume using the FAT32 file system so that the FAT ends up larger than 16 MB less 64 KB in size.
• You cannot format a volume larger than 32 GB in size using the FAT32 file system in Windows 2000. The Windows 2000 FastFAT driver can mount and support volumes larger than 32 GB that use the FAT32 file system (subject to the other limits), but you cannot create one using the Format tool. This behavior is by design. If you need to create a volume larger than 32 GB, use the NTFS file system instead.

Keep in mind the above is Windows (Microsoft). Linux and Android handle things a little differently. All of the Linux filesystem drivers support all three FAT types, namely FAT12, FAT16 and FAT32. As far as I know Android should handle them the same way. Other common features that they all support are various Linux mounting options (specified with the -o option to the mount command), such as "uid", "gid", "umask", etc.

There really is so much more to partitioning and formatting that I simply cannot cover it all here, however, all the information is already available someplace on the internet. Search is a wonderful tool, everyone!


SOME OTHER SD CARD RELATED STUFF TO PONDER


Finding and Fixing (?) Bad Blocks

SD cards include controller circuitry to perform bad block management and wear leveling. Although there is software to find bad blocks considering the price of these cards today if you're starting to see bad blocks the device is just plain going bad. Time to get a new one before you have serious problems.

You could run diagnostics to try and find/fix it if you really want to. This can be done on your PC and a card reader or in Android Terminal:


For ext2/3/4 partitions you can use e2fsck but it needs to run on an unmounted partition. If this is not clear you probably should not attempt it until you read a lot more, sorry.


You cannot fix bad blocks. Once a block is bad - it remains bad. Kind of like trying to fix a battery that cannot hold a charge any longer.

There are many supposed fixes claiming to do magic. None actually work. You may be able to tell the system a block is bad and it should not be used, thus avoid problems, but there is no fix.

Eventually, when your device develops too many bad blocks (around 50% of NAND) it will go to join that big phone company in the sky - time to get a new phone.


The best way to check bad blocks is to format all partitions in ClockWorkMod (warning: this will wipe your phone completely!) and check the recovery log when finished - after the format go to Advanced in ClockWorkMod Recovery and choose report error, this will save your log in your /sdcard/clockworkmod You can view this file with most text editors or copy/email it to your computer for viewing.

Alternately, you can try to reboot your device. When the boot process is completed (before you do anything else) load the TERMINAL program and enter the following:

Code:

su
dmesg > /sdcard/dmesg.txt

This will create a text file called dmesg.txt in the root of your card which can be viewed with a text editor. Look for the kernel name early in the file. Bad blocks, if any, should be listed not too far after the name. Or try Lumberjack.

Thanks to Robbie P for the reminder on a HD2/MAGLDR option:

Robbie P said:

If using MAGLDR, go to services/DMESG to SD to get Dmesg from last boot.​

Click to expand...

Click to collapse

Since we seem to be talking about bad blocks - it has been some time since this info was originally posted. For everyone who missed it (or forgot), here it is again.

The HD2 aka Leo is manufactured with 2 different NAND Flash ROM chipsets.
You can find the type by entering the tri-color boot mode.

If you see PB81120 SS-B3 on the first line that means your NAND Flash ROM is from Samsung (KBY00U00VM)

​

If you see PB1120 HX-B3 on the first line your NAND Flash ROM is from Hynix (H8BFS0WU0MCR)

​

After about one year of usage the Samsung chipset usually has a dozen or so bad clusters.
Hynix normally has none of them or just a couple.




A quick way to delete the Dalvik cache from your ext4 partition.

There are many, many ways to get this done should it be desired. One quick and easy way to do it in Android Terminal is:


Reboot your device and wait for the boot process to rebuild the cache.


Checking the system logs for problems

Problems with SD Cards as well as NAND and applications are often recorded in one of the Android system logs. Developers are often asking for logs so they can identify the source of a complaint. While not difficult to get, these logs are not in plain view. One the quick and easy ways to view and save the Android logs is a utility called Lumberjack.

The free Lumberjack software is available here.



If you prefer to use Android Terminal you can dump a lot of this info into text files which you can view later or email. The following will dump DMESG, KMSG and LOGCAT to your SD Card:





Comments? Additions? Suggestions? They are all welcome.
Flame wars (relating to SD Cards or otherwise) are not. :-]

HOW TO identify counterfeit Secure Digital Cards -and- backup your card to your PC!

IDENTIFYING COUNTERFEIT CARDS

A significant number of buyers have been deceived by inferior quality, cheap, slow SD cards rebadged as SanDisk or other brand names for quick profit. Many buyers get scammed by fraudulent sellers and products each day. There are a number of blogs showing images of fake cards.

This post is intended to help those who already have the card and would like to be certain.

With flash memory being able to be manipulated into displaying a set or upgraded fake capacity, there must be a way to efficiently test the flash memory. Not only is this testing critical for end users, it is essential for product manufacturers further up the supply chain to have a reliable way to detect fake Nand flash, otherwise they will be unwittingly producing products with incorrect capacities and creating marketplace chaos as well as soiling their reputation. The industry standard for testing memory is the burn in test, which essentially writes a set amount of data onto the memory, and then verifies said data. Errors signify that the memory is unstable and of lower quality (downgrade), or possibly that the memory has been upgraded to a fake capacity.

By far the most widely used and long-standing champion of burn-in testing for Nand flash is the h2testw.exe program, or affectionately known as the H2 burn-in test. Other benchmark and burn in testing programs have come and gone, all defeated by the upgraders in China. H2 was originally written by Harald Bogenholz for c't Magazin (Magazin für Computertechnik), a German computer magazine, and has been used extensively in the flash memory industry from China, Taiwan, to Korea, since 2008. The same version 1.4 has been in use since 2008 and has never been updated, which just goes to prove the reliability of Mr Bogenholz's awesome burn-in testing program.

Here's how:


If there are errors, chances are the flash is faulty or has been upgraded to a fake capacity.


As long as the flash IC has been attached onto a PCB with a controller, it is possible the memory has already been faked. Each time flash memory exchanges hands post SMT production, from half finished PCBA, to finished product QC, or the end distributor's inspection; H2 is there every step of the way to verify the memory capacity. Hopefully the champ can continue to defeat all of the most sophistaced hackers and upgraders to ensure everyone gets the memory capacity that they paid for.






BACKING UP YOUR SD CARDS

For those of us who have only Windows systems backing up the ext4 partition with the rest of the card has been a bit of a chore. Not any longer! Thanks to MarkAtHome for finding W32ImageWriter we can now create a complete image backup of the card - all partitions - and restore it in tact! The following is a step-by-step of how I did it.


Win32DiskImager is a freeware project currently in beta release v0.6. It will create and restore an image copy of your entire card to and from your Windows PC without a Linux system. Note that image copy means a sector by sector copy of the card into an image file. The program does not care what is on the card!

You can read more about it here (ignore the Rasberry Pi headings!) - I am not going to duplicate their entire page here, sorry.


Once you download the binary (link below) you can unzip it into any folder of your choice. It does not require "installation". The unzipped content of the download should look like this:




Place your SD Card into a USB card reader - it should have a Windows drive letter assigned to it. Now simply launch W32DiskImager.exe on your PC. The card I was using in my phone was a SanDisk Class 4 32 GB card. Looking at it in Minitool showed the following information. You can see my FAT32 and ext 4 partitions:




Keep in mind that this is beta - it still has some minor issues. Upon launch I received an error which may have been caused by Windows Explorer still being open. It seems this app expects exclusive control of the card although clicking OK continued operation without any further issues.




Once at the main screen of the app there is a "read" and "write" button choice. Select a folder and a file name on your PC for the image you are about to create and click "read" to copy the content of your card to your PC. The program prefers a file extension of .img.




This process will take a while since the entire card is being copied sector by sector, not only the files on the card - be patient. When the process completes you will have a large image file of your entire SD card - roughly the size of the card (again, not the content). To test the process I used a second SanDisk Class 4 32 GB card. It was clean formatted with FAT32 without any additional partitions.

I launched W32ImageWriter again, located the image file I created above, made sure the correct drive letter was shown for my card and clicked "write" to copy the saved image from the PC to the card. This process took considerably longer than the creation process since write speeds to the cards are slower than read speeds. Leave it alone, go get some coffee or tea and relax. If you watch the progress bar you will slow the process. When the restoration completed I looked at the card in Minitool and compared it to the first card - I could not see a difference.




The final test was to put the second card into my phone and power up. All was fine, as before. Just to be certain I didn't mix things up and shut the phone down and inserted the other (the original, I think) card and powered up again. As far as the phone was concerned the cards were identical.

And, I now had a complete card image saved on my PC, including the FAT32 and ext4 partitions, which I could recreate any time on the same or another card! Very cool!



If you'd like to give it a try you can find the free binary here.
I would appreciate hearing about your experience with it if you try it.

lucubrb said:

i'm using Transcend 16GB Micro-SDHC (Class10)

and using swapper2, SD Card Boost

Click to expand...

Click to collapse

About swappers....

Swapper2 is an app that lets you run a virtual memory on your sd card instead of using the internal phone memory. All swappers will decrease the life of your card (or NAND) simply by their nature, but SD cards are pretty cheap these days.

The intent of swapping is to create "make believe" RAM for working processes by moving applications in and out of working memory.

Unlike traditional swap, Android's Memory Manager kills inactive processes to free up memory. Android signals to the process, then the process will usually write out a small bit of specific information about its state (for example, Google Maps may write out the map view coordinates; the browser might write the URL of the page being viewed) and then the process exits. When you next access that application, it is restarted: the application is loaded from storage, and retrieves the state information that it saved when it last closed. In some applications, this makes it seem as if the application never closed at all. This is not much different from traditional swap, except that Android apps are specially programmed to write out very specific information, making Android's Memory Manager more efficient than swap.

Personally, I much prefer tweaking the Memory Manager settings than using swap files.

About SD Card Boost...

Not sure what this is. Is this something like the read buffer or something else?

A little improvement for you

Android has numerous ways to tweak performance - way too many for a single post. There is absolutely no substitute for reading and learning more about the internals of various Android systems. Short of writing a book (no plans for that!) I shall not attempt to explain them all or even most. Some of them, when applied incorrectly, will make your device perform poorly or even stop it from functioning (bootloops, no boot, etc.)

Note that these ideas are not specific to an installation type (e.g. NAND only, DataOnExt or NativeSD) but rather to using an SD Card with your installation.

Please keep in mind that these suggestions will not completely eliminate lags, especially during startup. Android has to do what Android has to do. Likewise, if you are using a relatively old device (like the HD2 that I am using) it cannot act completely like the latest technology. That is just reality.

Don't forget that no matter what you do to your device and how much you tweak it if you are using a sub-par SD Card (and there are many of those) not much else will matter or help.

Before you do anything else make sure your card's FAT partition was formatted with SD Format to get the best cluster size and format for it. Of course, if you have data on the card it'd be a good idea to back it up first.

I will only get into two of the simple and relatively safe methods here. They can make a significant impact on your system.

Since this topic is about SD Cards it assumes you are using an SD Card for apps and/or data and/or Dalvik... Two quick and easy ways to improve your device's response involve read buffers (already mentioned here) and your device's Auto Memory Manager (AMM). Both of the settings described here are lost after a reboot if they are not re-applied!


A read buffer can be set for your card via shell script (init.d support required) or by various apps, such as ROM Toolbox. I will use the latter for illustrations aiming this for people relatively new to Android. If you prefer to do things another way I agree there are a number of other ways to do it.


If your buffer is too small it will accomplish nothing. As an example, using a 256KB buffer with a 32KB cluster size it will yield a factor of 8 reading clusters! I suspect that is not very efficient. Setting a 16384KB read buffer will increase the same operation to a factor of 512 - saving nearly 500 cluster reads. The resulting performance increase can be dramatic. Please remember to experiment - while a large buffer may do wonders (as it did for me) it is not always the best choice. See this post for a little contrast.


The read buffer is, unfortunately, far from being the sole factor when it comes to device performance. As I already said, I will not go into all of them but a second one that can impact application load times and "lag" is Android's Auto Memory Manager which controls how much free memory is forced by the OS (MinFree). One of the simple ways to set it is once again ROM Toolbox. Try the settings in this image, along with a (relatively) large read buffer, reboot your system and let things settle down. You might just find your device's response a little better.



Incidentally, experimenting with these settings is not very likely to do any harm to your system as long as you do not pick and specify random (unrealistic) values. Most apps that allow you to alter these settings will use relatively safe numbers. Trying different values ( at least for a few hours each) may just tell you which setting is best for your device and improve your overall experience quite a bit.