[INFO]What is EpicMTD, and what you need to know.

Search This thread

toadlife

Inactive Recognized Developer
Aug 19, 2008
1,208
1,012
Lemoore, CA
I'm not sure why "reboot to recovery" doesn't work yet. As I understand, executing "reboot recovery" from adb or the command-line does work, although I've not verified that behavior myself. There's some bug that must prevent the command from working properly when executed from the UI, that needs to be investigated yet.

`reboot recovery` from adb/shell and rebooting to recovery from ROM Manager doesn't work for some reason. It just reboots. Right now, the only way I've been able to get into recovery while on MTD is to do the three finger salute.


Thanks for your post BTW. It helped me figure out what parts I needed to get my kernel working with MTD.
 

KennyG123

Senior Moderator / Mod Committee / Spider-Mod
Staff member
Nov 1, 2010
39,957
54,785
Right behind you!
Partition sizes are set in the kernel. I imagine this can lead to all kinds of hilarity if someone flashes a ROM/kernel with one set of partition sizes and then flashes another MTD kernel with different partition sizes.

Wow, that does not sound cool. Maybe all the DEVs can agree on size standard for that.
 

codest3r

Senior Member
Oct 27, 2010
348
58
Orlando, FL
Is there anything that needs to be done to make a theme work on MTD/YAFFS2?

And what determines the partition sizes? Kernel, ROM, or other?

Assuming the recovery kernel has the drivers loaded and can mount the mtd partition flashing filesystem changes and overwrites should be the same, as I understand theming pushes work.

Other. Neither the ROM or kernel would set the initial size. The partition map would. It's not clear to me if the values (for size, location, and order) are set in a config file or script during conversion from BML to MTD.

The best visual tool I could use as an analogy would be gparted in linux or disk management (microsoft management console snapin) in windoze. Once you set partitions with said tools of various sizes, start points, end point, format types (ext, fat, ntfs, etc), you can load any number of OS's on each respectively (or ROM's in our case) assuming they are compatible with the filesystem types and have a common unifying bootloader that understands them all (that can read each fs and call the respective bootloader). Sorry for the poor anaology but trying to put it all in more familiar terms.

Sent via Decadent Epic 4G CM7
 
  • Like
Reactions: mrcollpepper

toadlife

Inactive Recognized Developer
Aug 19, 2008
1,208
1,012
Lemoore, CA
Wow, that does not sound cool. Maybe all the DEVs can agree on size standard for that.

In testing with my ROM, I modified the updater script to check the /system and /data partition sizes and if they don't match up, flash the kernel and reboots before flashing the ROM.

A standard partition size would be easiest, but a checking partition sizes would also work.

---------- Post added at 02:51 PM ---------- Previous post was at 02:44 PM ----------

Other. Neither the ROM or kernel would set the initial size. The partition map would. It's not clear to me if the values (for size, location, and order) are set in a config file or script during conversion from BML to MTD

I don't think that's true. I'm pretty sure the partition map is defined in the kernel...period. So if you write data to your partitions to the nand and then load a kernel with a different partition map, the device will not boot because it won't be able to read the partitions.

I'm not 100% sure about this...only 98%, so if someone wants to correct me, please do.
 

codest3r

Senior Member
Oct 27, 2010
348
58
Orlando, FL
I don't think that's true. I'm pretty sure the partition map is defined in the kernel...period. So if you write data to your partitions to the nand and then load a kernel with a different partition map, the device will not boot because it won't be able to read the partitions.

You do need the proper drivers in the kernel to interact with the OneNAND, it's partitions, and data formats (e.g. the storage memory or ROM in this case). However, the kernel itself doesn't define the partition map. Without the proper kernel (and associated drivers) we can't mount the filesystems in storage memory, that's for sure!

With BML the partitions are sized and created from the Partition Information Table during flash (e.g. ODIN, Samsung Upgrade Tool, Heimdall). More information on PIT specifics can be found (with good links too) in a post from the SGS i9000 forum: [GUIDE] [INFO] The reality of PIT files

Notice in his post the PIT is not part of the kernel - it has it's own parition. Which is likely why you don't have to specify the PIT file when flashing BML ROM's on our Epic because it just uses the PIT on the device! Pretty genius idea to keep it stored here in it's own space.

START
PBL (256KB: Primary Boot Loader: boot.bin)
PIT (*256KB: .pit )
EFS (5.9MB: efs.rfs )
SBL (1.25MB: Secondary Boot Loader: Sbl.bin)
SBL2 (1.25MB: Backup Secondary Boot Loader: Sbl.bin)
PARAM (*640KB: param.lfs)
KERNEL (7.5MB: Primary Kernel: zImage)
RECOVERY (7.5MB: Backup Kernel: zImage)
FACTORYFS (276.3MB: factoryfs.rfs)
DBDATAFS (126.7MB, 117.2MB, 107.2MB (depends on pit file): dbdata.rfs)
CACHE (30.1MB: cache.rfs)
MODEM (12MB: modem.bin)
END
*=Assumed


I'm not sure what they are using with MTD to set the partition map. It could be set in the conversion script or in a configuration file. I couldn't find it after a cursory skim of the contents of the CWM flashable zip. That has eluded me so far. :)
 

toadlife

Inactive Recognized Developer
Aug 19, 2008
1,208
1,012
Lemoore, CA
However, the kernel itself doesn't define the partition map....I'm not sure what they are using with MTD to set the partition map. It could be set in the conversion script or in a configuration file. I couldn't find it after a cursory skim of the contents of the CWM flashable zip. That has eluded me so far. :)

The reason you couldn't find it is because it's defined by the kernel. Specifically, the partition layout is defined in the file Kernel/drivers/mtd/onenand/samsung_epic.h

See: https://github.com/toadlife/CleanGB-Kernel/blob/mtd/Kernel/drivers/mtd/onenand/samsung_epic.h
 

codest3r

Senior Member
Oct 27, 2010
348
58
Orlando, FL
The reason you couldn't find it is because it's defined by the kernel. Specifically, the partition layout is defined in the file Kernel/drivers/mtd/onenand/samsung_epic.h

See: https://github.com/toadlife/CleanGB-Kernel/blob/mtd/Kernel/drivers/mtd/onenand/samsung_epic.h

Ahhh... ha. Thanks for that! I'm surprised to see that there. :eek:

Leads me to more questions! :eek:

If you were to change the mapping (i.e. modify offset or size of a few partitions in samsung_epic.h) - compile and flash your kernel - what would happen? Assuming you didn't change ROM's - just kernels. My guess is nothing if your phone is already converted to MTD. The kernel should mount the partitions just fine without the map, right? :confused:

However, if your device wasn't converted to MTD yet the flashable ZIP for a ROM (with kernel) would contain the script instruction to check and if not converted -> execute conversion... at which point it would look for these values in the kernel. Right?

Taking that a step further, each ROM maker could choose whether to check the sizes of MTD partitions and if in tolerance (some predefined range of acceptable) it would flash ROM package on existing partition sizes, OR if not acceptable it could wipe out the partition (and all data on it) and start over with preferred sizes. Right?

Debunk my guesses. Thanks for the learning opportunity. Please excuse the ignorance and dumb questions - I'm trying to figure out how this works and missed the instruction manual. :D
 

toadlife

Inactive Recognized Developer
Aug 19, 2008
1,208
1,012
Lemoore, CA
If you were to change the mapping (i.e. modify offset or size of a few partitions in samsung_epic.h) - compile and flash your kernel - what would happen? Assuming you didn't change ROM's - just kernels. My guess is nothing if your phone is already converted to MTD. The kernel should mount the partitions just fine without the map, right? :confused:

I just tested it out. I flashed my ROM using this kernel with this partition layout...

Code:
Filesystem                Size      Used Available Use% Mounted on
/dev/block/mtdblock4     25.0M      2.3M     22.7M   9% /cache
/dev/block/mtdblock3    695.0M    407.0M    288.0M  59% /data
/dev/block/mtdblock2    250.0M    206.2M     43.8M  82% /system

I then flashed a kernel with the following partition layout...

Code:
Filesystem                Size      Used Available Use% Mounted on
/dev/block/mtdblock4     50.0M      2.3M     47.7M   5% /cache
/dev/block/mtdblock2    250.0M    206.2M     43.8M  82% /system
/dev/block/mtdblock3    670.0M    407.0M    263.0M  61% /data

...and the partitions mounted just fine, with different sizes!

Unfortunately, after flashing Tortell's MTD kernel which has the "stock" partition layout (a different size for /system), I got this message when trying to mount system partition:

Code:
Failed to mount /dev/block/mtdblock2 on /system: Invalid argument

/data and /cache still mounted.

Flashing back the original kernel allowed my to mount to all partitions again.


However, if your device wasn't converted to MTD yet the flashable ZIP for a ROM (with kernel) would contain the script instruction to check and if not converted -> execute conversion... at which point it would look for these values in the kernel. Right?

No, it just checks to see if the device is MTD or BML. If it is BML, it flashes the MTD kernel and reboots. I added a part to that script which checks the partition sizes and if they don't match up with what they should be, it does the same thing - flashes the kernel and reboots. I inserted the code into updater.sh. This could be put into a seperate shell script. Here is the code. It's adapted from the code that checks for BML/MTD.

Code:
# check to see if partition sizes are correct
/tmp/busybox mount -t yaffs2 /dev/block/mtdblock2 /system
/tmp/busybox mount -t yaffs2 /dev/block/mtdblock3 /data
if (! df /dev/block/mtdblock2 | grep "256000") || (! df /dev/block/mtdblock3 | grep "711680"); then
# Our partition sizes are not correct

# unmount, format and mount system
/tmp/busybox umount -l /system
/tmp/erase_image system

# unmount and format cache
/tmp/busybox umount -l /cache
/tmp/erase_image cache

# unmount and format datadata
/tmp/busybox umount -l /data
/tmp/erase_image userdata

# make sure sdcard is mounted
if ! /tmp/busybox grep -q /mnt/sdcard /proc/mounts ; then
/tmp/busybox mkdir -p /mnt/sdcard
/tmp/busybox umount -l /dev/block/mmcblk0p1
if ! /tmp/busybox mount -t vfat /dev/block/mmcblk0p1 /mnt/sdcard ; then
/tmp/busybox echo "Cannot mount sdcard."
exit 1
fi
fi

# remove old log
rm -rf /mnt/sdcard/cyanogenmod_bml.log

# everything is logged into /sdcard/cyanogenmod.log
exec >> /mnt/sdcard/cyanogenmod_bml.log 2>&1

# write the package path to sdcard cyanogenmod.cfg
if /tmp/busybox test -n "$UPDATE_PACKAGE" ; then
PACKAGE_LOCATION=${UPDATE_PACKAGE#/mnt}
/tmp/busybox echo "$PACKAGE_LOCATION" > /mnt/sdcard/cyanogenmod.cfg
fi

# Scorch any ROM Manager settings to require the user to reflash recovery
/tmp/busybox rm -f /mnt/sdcard/clockworkmod/.settings

# write new kernel to boot partition
/tmp/flash_image boot /tmp/boot.img
if [ "$?" != "0" ] ; then
exit 3
fi
/tmp/busybox sync

/sbin/reboot now
exit 0
fi

Taking that a step further, each ROM maker could choose whether to check the sizes of MTD partitions and if in tolerance (some predefined range of acceptable) it would flash ROM package on existing partition sizes, OR if not acceptable it could wipe out the partition (and all data on it) and start over with preferred sizes. Right?

Exactly.

---------- Post added at 12:14 PM ---------- Previous post was at 12:04 PM ----------

Let me add one very important note. After flashing those different kernels and mounting partitions with them, when I finally booted my phone, it booted fine, but my home screen layout has reverted to stock layout, which suggests that some of my data was munched when I mounted /data using different sizes. My downloaded apps still show up in my launcher, so some of my data is still there, but obviously some of it was lost.

I think it's safe to say that if partition layouts are changed, partitions will need to be erased and recreated.
 

codest3r

Senior Member
Oct 27, 2010
348
58
Orlando, FL
Let me add one very important note. After flashing those different kernels and mounting partitions with them, when I finally booted my phone, it booted fine, but my home screen layout has reverted to stock layout, which suggests that some of my data was munched when I mounted /data using different sizes. My downloaded apps still show up in my launcher, so some of my data is still there, but obviously some of it was lost.

I think it's safe to say that if partition layouts are changed, partitions will need to be erased and recreated.

First, thanks for taking the time to run this through the gamut and test it out! Also appreciate you sharing the results and code. Helps me understand more of whats happening. Thanks man. :D

Fascinating to me. Seems like a risky approach to map the partitions in the kernel and not have it fixed elsewhere, but I assume this is best solution for employing MTD. Wonder if any other approaches are available or were considered? Frankly, I don't want a kernel modifying the partition sizes, locations, etc. -- at least without me being fully aware of it before flashing. Hopefully all the kernel devs are aware of this!

And I agree with your conclusion. If you had modified the start, stop, or size parameters of the system partition, or even tried to swap the /data and /system locations, it likely might have prevented it from even booting up.

That said, it should be possible to build on your code that recognizes the different sizes (as you did) and if changes were warranted to partition layout it could first verify the contents of each partition would fit in the new partition layout and if so backup the system, data, and cache (easiest to just straight up copy contents to /sdcard instead of nandroid?), then execute the partition layout changes, then after changes are made to the partition sizes restore files from backup. I suppose... ;-)
 

noobnl

Inactive Recognized Developer
Jun 23, 2010
908
3,672
Maybe you shouldn't poke around with partition map.
What wrong with the one we have now? Please don't mess around with our layout. Why do you need 50 MB of Cache space?

market fix: https://github.com/EpicCM/android_d...mmit/a7b74d44929150c24ad61daa3f7f8a93e954d5ce
https://github.com/EpicCM/android_v...m-rebase/prebuilt/common/etc/init.d/06mountdl
Hope that helps!
I just tested it out. I flashed my ROM using this kernel with this partition layout...

Code:
Filesystem                Size      Used Available Use% Mounted on
/dev/block/mtdblock4     25.0M      2.3M     22.7M   9% /cache
/dev/block/mtdblock3    695.0M    407.0M    288.0M  59% /data
/dev/block/mtdblock2    250.0M    206.2M     43.8M  82% /system

I then flashed a kernel with the following partition layout...

Code:
Filesystem                Size      Used Available Use% Mounted on
/dev/block/mtdblock4     50.0M      2.3M     47.7M   5% /cache
/dev/block/mtdblock2    250.0M    206.2M     43.8M  82% /system
/dev/block/mtdblock3    670.0M    407.0M    263.0M  61% /data

...and the partitions mounted just fine, with different sizes!

Unfortunately, after flashing Tortell's MTD kernel which has the "stock" partition layout (a different size for /system), I got this message when trying to mount system partition:

Code:
Failed to mount /dev/block/mtdblock2 on /system: Invalid argument

/data and /cache still mounted.

Flashing back the original kernel allowed my to mount to all partitions again.




No, it just checks to see if the device is MTD or BML. If it is BML, it flashes the MTD kernel and reboots. I added a part to that script which checks the partition sizes and if they don't match up with what they should be, it does the same thing - flashes the kernel and reboots. I inserted the code into updater.sh. This could be put into a seperate shell script. Here is the code. It's adapted from the code that checks for BML/MTD.

Code:
# check to see if partition sizes are correct
/tmp/busybox mount -t yaffs2 /dev/block/mtdblock2 /system
/tmp/busybox mount -t yaffs2 /dev/block/mtdblock3 /data
if (! df /dev/block/mtdblock2 | grep "256000") || (! df /dev/block/mtdblock3 | grep "711680"); then
# Our partition sizes are not correct

# unmount, format and mount system
/tmp/busybox umount -l /system
/tmp/erase_image system

# unmount and format cache
/tmp/busybox umount -l /cache
/tmp/erase_image cache

# unmount and format datadata
/tmp/busybox umount -l /data
/tmp/erase_image userdata

# make sure sdcard is mounted
if ! /tmp/busybox grep -q /mnt/sdcard /proc/mounts ; then
/tmp/busybox mkdir -p /mnt/sdcard
/tmp/busybox umount -l /dev/block/mmcblk0p1
if ! /tmp/busybox mount -t vfat /dev/block/mmcblk0p1 /mnt/sdcard ; then
/tmp/busybox echo "Cannot mount sdcard."
exit 1
fi
fi

# remove old log
rm -rf /mnt/sdcard/cyanogenmod_bml.log

# everything is logged into /sdcard/cyanogenmod.log
exec >> /mnt/sdcard/cyanogenmod_bml.log 2>&1

# write the package path to sdcard cyanogenmod.cfg
if /tmp/busybox test -n "$UPDATE_PACKAGE" ; then
PACKAGE_LOCATION=${UPDATE_PACKAGE#/mnt}
/tmp/busybox echo "$PACKAGE_LOCATION" > /mnt/sdcard/cyanogenmod.cfg
fi

# Scorch any ROM Manager settings to require the user to reflash recovery
/tmp/busybox rm -f /mnt/sdcard/clockworkmod/.settings

# write new kernel to boot partition
/tmp/flash_image boot /tmp/boot.img
if [ "$?" != "0" ] ; then
exit 3
fi
/tmp/busybox sync

/sbin/reboot now
exit 0
fi



Exactly.

---------- Post added at 12:14 PM ---------- Previous post was at 12:04 PM ----------

Let me add one very important note. After flashing those different kernels and mounting partitions with them, when I finally booted my phone, it booted fine, but my home screen layout has reverted to stock layout, which suggests that some of my data was munched when I mounted /data using different sizes. My downloaded apps still show up in my launcher, so some of my data is still there, but obviously some of it was lost.

I think it's safe to say that if partition layouts are changed, partitions will need to be erased and recreated.
 

toadlife

Inactive Recognized Developer
Aug 19, 2008
1,208
1,012
Lemoore, CA
Maybe you shouldn't poke around with partition map.
What wrong with the one we have now? Please don't mess around with our layout. Why do you need 50 MB of Cache space?

market fix: https://github.com/EpicCM/android_d...mmit/a7b74d44929150c24ad61daa3f7f8a93e954d5ce
https://github.com/EpicCM/android_v...m-rebase/prebuilt/common/etc/init.d/06mountdl
Hope that helps!

Yeah I know of that fix. More space for /data is actually what I might be interested in. Good luck with your partition size mandate. Honestly there is NO point to using MTD/Yaffs2 if we don't make use of the ability to change partition sizes.
 

noobnl

Inactive Recognized Developer
Jun 23, 2010
908
3,672
Yeah I know of that fix. More space for /data is actually what I might be interested in. Good luck with your partition size mandate. Honestly there is NO point to using MTD/Yaffs2 if we don't make use of the ability to change partition sizes.

I think you should keep cache the same size but only change data and system because it is better to change only one offset instead of two offset.
 

hazridi

Senior Member
Dec 31, 2010
89
21
There's plenty of reason to move to MTD without altering the current MTD partition layout, like getting rid of the crappy BML abstraction layer and replacing it with one actually built to reduce issues with flash filesystems, so we can use yaffs2 and have more advanced wear levelling. And reduce the reliance on a closed source filesystem and block layer.

Agree with noob here, you're really not creating anything but a huge problem for users.
 

toadlife

Inactive Recognized Developer
Aug 19, 2008
1,208
1,012
Lemoore, CA
There's plenty of reason to move to MTD without altering the current MTD partition layout, like getting rid of the crappy BML abstraction layer and replacing it with one actually built to reduce issues with flash filesystems, so we can use yaffs2 and have more advanced wear levelling. And reduce the reliance on a closed source filesystem and block layer.

Agree with noob here, you're really not creating anything but a huge problem for users.

You are right that this will cause issues for users, especially those who love the kernel/ROM hop, but how will those issues be avoided when ICS/CM9 rolls out and more space is needed in system? Will another official partition layout be shouted down from the mountain top and those of us who might want to continue to build TW based ROMs be forced to use massive system partitions which will go only half-used?
 

codest3r

Senior Member
Oct 27, 2010
348
58
Orlando, FL
I think you should keep cache the same size but only change data and system because it is better to change only one offset instead of two offset.

Would it be possible to merge system and data on a common root partition? No semi-arbitrary boundaries. Anything not used by ROM/system is avail for data. Still leave something tiny for cache like you have now.

I know this can cause problems with problem/offending apps bugging out and hogging space beyond reason or debugging logs running unchecked.... But that's unlikely to be an issue for us right?

Sorry if it's a dumb question but thought it would eliminate any differences of opinion (or need) for different ROM's partition sizing allocations on MTD. Probably some dependencies I'm not aware of... ;-)
 

toadlife

Inactive Recognized Developer
Aug 19, 2008
1,208
1,012
Lemoore, CA
Would it be possible to merge system and data on a common root partition? No semi-arbitrary boundaries. Anything not used by ROM/system is avail for data. Still leave something tiny for cache like you have now.

I know this can cause problems with problem/offending apps bugging out and hogging space beyond reason or debugging logs running unchecked.... But that's unlikely to be an issue for us right?

Sorry if it's a dumb question but thought it would eliminate any differences of opinion (or need) for different ROM's partition sizing allocations on MTD. Probably some dependencies I'm not aware of... ;-)

I don't see that as a viable option. It would require a radical change in the way partitions are arranged and it would mean /system has to be mounted read+write, whereas it's typically mounted read-only.

I think I'm going to stick with the CM7 defaults for now. With the way people like to hop from kernel to kernel, and the way people don't read or follow directions, it would cause too much havoc for users. It would be nice to have the partition layouts defined somewhere else, so the kernel wouldn't have to have them baked in.
 

mkasick

Retired Recognized Developer
Aug 10, 2009
470
830
Seems like a risky approach to map the partitions in the kernel and not have it fixed elsewhere, but I assume this is best solution for employing MTD. Wonder if any other approaches are available or were considered?
There is an on-flash partition table, bml2 (not mapped in mtd), and it would certainly make sense to support it under MTD kernels to allow for different partition sizes. It's not done simply becuase there's no MTD-pathway code for reading the on-flash partition table. Partition sizes are hardcoded in the Samsung MTD driver on the Nexus S, and, since that's how the driver was written, was kept for Galaxy S MTD kernels.

It's possible to add on-flash partition table support for MTD, but would require some amount of code. I'd guess it's a reasonably-sized project for someone sufficiently ambitious. The format is known for the relevant portion of the partition table, plus there's code in the tfsr driver for reading it also--which I wouldn't copy, but might help in making sense of it.

Honestly there is NO point to using MTD/Yaffs2 if we don't make use of the ability to change partition sizes.
To be honest, the main benefit of MTD vs BML is maintenance of the CM port. Since all other supported SGS devices use MTD, for the Epic to use MTD reduces the amount of our device-specific configutation. While a BML-based CM7 port is certainly doable (an has been done), being the sole BML device means there's more configuration junk that we have to maintain.

The other major benefit of MTD would be the ability to forward port victory (Epic) support to a Nexus S-derived (S5PV210) ICS kernel, i.e., for CM9. With Samsung's proprietary storage stack, we're stuck on our present kernel version unless newer modules leak out somewhere. Given that Samsung appears uninterested in porting ICS to the SGS, those might be hard to come by, especially timely.

Personally I don't think MTD is particularly advantageous to TW kernels, aside from where basing on the EpicCM kernel makes such kernels easier to maintain.
 
Last edited:

Top Liked Posts

  • There are no posts matching your filters.
  • 101
    Getting started

    Before I get into the thick of this, I've been for a little over a week or maybe even more taking as much time as I can to explain to people what MTD is and the risks of it. And unfortunately most people just by natural human tendencies have an issue comprehending it which believe me is perfectly fine haha, it's a lot to take at once. So I'm going to layout the basic need to knows, and if any more information is needed feel free to pm me and I'll add it into this main post.


    I'm going to try to make this as simplistic as I can.:x


    What is a partition?

    A partition is an area of allocated space, a division of the whole overall area of space. In this case our partitions on the Epic 4G are /System, /Data, as well as /Cache. All with set permanent sizes.

    What is a partition map?

    A partition map is the configuration of our partitions, it's what in a vagueness sets our required sizes for the divisions of our nand also known as flash memory. A partition or partition map should not be confused with a file system. An example would be BML and MTD.

    What is a file system?

    A file system resides on the partition map and governs the data being read/wrote/moved/etc by the Operating System, in this case Android. Changing a file system is less complex than an overall change in partition mapping. They again, are not the same thing.

    What is MTD?

    MTD is an Open Source Partition map. It allows those who are using it control over how their partitions are sized and how much space is allocated here and how much space is taken away from there. Currently on MTD we have 689 megabytes of space allocated to our /data partition allowing more to be downloaded from the market as an example. MTD as a partition config has YAFFS2 as a file system residing on it governing how data is transferred and the speed of which it is done. EXT2 through 4 aren't possible on the MTD platform, just as YAFFS2 may not be possible on the BML proprietary platform.

    What is BML?

    BML like MTD is a partition map, however it is proprietary in nature, Close Source if you will. The size for /System /Data /Cache is set and permanent and makes opening up space more of a task for Developers. Stock the Epic 4G comes on BML, and is running RFS as it's file system, once rooted you can leave RFS for EXT4 (Journaled or Un-Journaled) as long as the kernel you use allows for EXT4. But in the end, changing a file system on BML does not lessen or enhance the control you have over your partitions.

    What do I need to know before flashing a rom with this?

    Currently there are two distributions which have moved over to the MTD platform.

    One of which and the first inital one being:

    [ROM] Epicmtd CM7 SELFKANG (12/10) - 2.3.7

    And the second which is a basis of the first:

    [ACS] [MTD] [YAFFS2] MIUI 1.12.2 Beta 6 MTD (Updated 12/10/2011)

    Within the flashable zips for these two roms there are scripts which completely format your device and move you to MTD. It is required to back up the things you wish to keep with Titanium backup if you have it. Nandroids from RFS/EXT4/BML do not work on MTD/YAFFS2. Alternatively if you have purchased appextractor or titanium back up pro licensing from the Android market you can make a nandroid before your move and then extract the data apps or system apps you want from that backup while on MTD/YAFFS2 without going into recovery.

    When flashing one of these ROMS in CWM5 your phone shall reboot during the installation if it finds you're on BML, don't panic it is a natural process of this move.


    When booting into recovery on MTD it reacts to how long you hold the 3 button combination. Being Volume down, Camera, and Power. Don't panic if it doesn't pop up, you must let go to let it know which path it goes to. If your phone looks like it's booting again don't do the combination. It's switching from the init to the recovery.

    CWM 5 is not broken, on MTD our kernel now has a 2 stage init. And it's part of the process.

    How do I go about flashing other roms if they're not MTD?

    Flashing a Stock TouchWiz rom can be very problematic, mainly due to the nature of and differences between MTD and BML. Currently there isn't a stock MTD kernel. To return to BML, you have a choice of either using Heimdall or Odin. To do so you require the victory.pit as well as a stock tar of your choice which includes our bootloader so that when you repartition your device all goes well without problem. There is no method to return without using Odin/Heimdall because BML is a proprietary configuration. We lack the tools to replicate Samsung's methods and mannerisms.


    What does it mean for me as an end user?

    As an End User, MTD is an opening to a new life for the Epic 4G. Things like ICS, more space in data or system, are more within our reach and grasp due to the nature of Open Source MTD is immersed in. We're closer to the Captivate, Fascinate, Vibrant, and Galaxy S international by being on MTD, we've that new freedom they've had for a long time. Not to say things like ICS aren't possible on BML but with this we're at a better standing point.


    Stock EI22 on MTD

    noobnl as well as Tortel have worked together to put a stock EI22 rom for the MTD platform which can be found here.

    [EI22] Stock-ish MTD Build

    This is for a stock experience and for those want to use MTD as well as make use of the stock features our phone has, nothing out of the ordinary or custom, just as a basic point. Rom Developers can go on towards moving their roms over to this platform now with the use of the boot.img but for now kernel developers will still have an issue nonetheless.



    Those responsible for this in no specific order: Decad3nce, noobnl, jt1134, mkasick, nullghost, nubecoder, DRockstar, UberPinguin, Rodderik, wtogami, as well as countless others.



    All things within this thread are subject to change if a need for correction is to be met.
    10
    I'm curious how the boot up process works with BML/RFS (or EXT4) vs MTD/YAFFS2.
    It's the same process until the kernel and initramfs is loaded. Specifically:
    • SoC ROM code loads and boots the Primitive Bootloader (PBL, boot.bin).
    • PBL searches the device partitions for the Secondary Bootloader (SBL, Sbl.bin), loads, and boots that.
    • SBL checks the "reboot mode" in param.blk (on stl6) and on a normal boot, loads the kernel stored in bml7 with init.rc, and on a recovery boot, loads the kernel stored in bml8. It also checks if the vol-down, camera, and power buttons are presed, and if so, sets the reboot mode for a recovery boot and loads the kernel stored in bml8.
    • The loaded kernel (stored on bml7 or bml8) also contains a initramfs, or root file system image. /init, in the initramfs checks the kernel command-line for a "bootmode" parameter set by SBL, which determines whether init.rc (normal boot) or recovery.rc/fota.rc (recovery boot) should be loaded.
    • As part of executing init.rc (or recovery.rc/fota.rc), the appropriate storage modules are loaded (fsr.ko, fsr_stl.ko, rfs.ko, rfs_glue.ko, j4fs.ko, and param.ko for a BML kernel), and various file-systems are mounted (stl6 as /mnt/.lfs, stl9 as /system, stl10 as /data, and stl11 as /cache for BML).

    An MTD kernel differs by the storage drivers used in Linux. Instead of the Samsung proprietary modules, we use the open-source OneNAND MTD driver. Now, our OneNAND chip is different/newer than the one used in other SGS devices, so we actually had to backport the "samsung_audi" driver from the SGSII kernel source tree. Amusingly, the SGSII doesn't use MTD either, but some SGSII development prototype devices did, so somewhere along the line Samsung added that driver to the kernel tree to support their newer OneNAND chips. This is one of the reasons why it took a while to support MTD on our device as the driver that ships with SGS and Nexus S kernels doesn't work for us.

    One of the takeaways from the earlier description of the Samsung boot chain is that both PBL and SBL must support I/O to the OneNAND chip, the OneNAND partition table, and the BML partition encoding. Additionally SBL must support STL for flashing via download mode, and for reading/writing param.blk. So, for MTD support, it's critical that retain compatibility with the partition table format, and use BML-encoded partitions for the kernel. Actually, the fact that we're running MTD is essentially invisible to the bootloader, it does the exact same booting sequence until the point that the kernel is booted and "we" have control.

    Where MTD differs is in the format of the /system, /data, and /cache partitions. Instead of using RFS (or ext4) on STL, on BML, we use yaffs2 straight on top of MTD. To flash the kernel, CyanogenMod uses a user-space tool, bml_over_mtd, to BML-encode a kernel and write it to the raw MTD device. That's basically it.

    I was hoping it would help me understand why reboot to recovery doesn't work on MTD yet and also why the 3 finger causes loops until you release it.
    I'm not sure why "reboot to recovery" doesn't work yet. As I understand, executing "reboot recovery" from adb or the command-line does work, although I've not verified that behavior myself. There's some bug that must prevent the command from working properly when executed from the UI, that needs to be investigated yet.

    As for the three-finger issue, that's an artifact of a fix that we have to use in order to boot into recovery on MTD kernels. The Fascinate, which has the same problem, uses a very similar fix.

    Earlier I mentioned that a normal vs recovey boot is determined by the "reboot mode" setting in param.blk. Specifically, when you do a three-finger boot, SBL sets "reboot mode" to boot recovery before loading bml8. Now, it's the responsibility of the recovery kernel to reset the reboot mode in parma.blk back to normal, otherwise SBL will always load bml8 and boot recovery on subsequent reboots/powerups, whether the vol-down, camera, and power buttons are held or not.

    Now, the problem is that param.blk is stored on a partition, stl6, that must remain BML/STL/j4fs encoded for bootloader compatibility, and so MTD kernels lack the driver support to mount that partition. Thus, we can't reset the reboot mode setting.

    So, the three-finger fix consists of a kernel with a stub initramfs installed to bml8. It's actually a BML kernel, that is, it supports Samsung's storage modules and not MTD. The sole job of this kernel is to reset the reboot mode in param.blk, poke a magic value into persistent memory, and reboot. On reboot, the MTD kernel in bml7 is loaded, checks for our magic value in persistent memory, and boot into recovery. The caveat is that, if you keep the vol-down, camera, and power buttons held, SBL will continue to boot bml8 repeatedly, which continues to reset the reboot mode and reboot the device.

    Hope that helps.
    4
    If you were to change the mapping (i.e. modify offset or size of a few partitions in samsung_epic.h) - compile and flash your kernel - what would happen? Assuming you didn't change ROM's - just kernels. My guess is nothing if your phone is already converted to MTD. The kernel should mount the partitions just fine without the map, right? :confused:

    I just tested it out. I flashed my ROM using this kernel with this partition layout...

    Code:
    Filesystem                Size      Used Available Use% Mounted on
    /dev/block/mtdblock4     25.0M      2.3M     22.7M   9% /cache
    /dev/block/mtdblock3    695.0M    407.0M    288.0M  59% /data
    /dev/block/mtdblock2    250.0M    206.2M     43.8M  82% /system

    I then flashed a kernel with the following partition layout...

    Code:
    Filesystem                Size      Used Available Use% Mounted on
    /dev/block/mtdblock4     50.0M      2.3M     47.7M   5% /cache
    /dev/block/mtdblock2    250.0M    206.2M     43.8M  82% /system
    /dev/block/mtdblock3    670.0M    407.0M    263.0M  61% /data

    ...and the partitions mounted just fine, with different sizes!

    Unfortunately, after flashing Tortell's MTD kernel which has the "stock" partition layout (a different size for /system), I got this message when trying to mount system partition:

    Code:
    Failed to mount /dev/block/mtdblock2 on /system: Invalid argument

    /data and /cache still mounted.

    Flashing back the original kernel allowed my to mount to all partitions again.


    However, if your device wasn't converted to MTD yet the flashable ZIP for a ROM (with kernel) would contain the script instruction to check and if not converted -> execute conversion... at which point it would look for these values in the kernel. Right?

    No, it just checks to see if the device is MTD or BML. If it is BML, it flashes the MTD kernel and reboots. I added a part to that script which checks the partition sizes and if they don't match up with what they should be, it does the same thing - flashes the kernel and reboots. I inserted the code into updater.sh. This could be put into a seperate shell script. Here is the code. It's adapted from the code that checks for BML/MTD.

    Code:
    # check to see if partition sizes are correct
    /tmp/busybox mount -t yaffs2 /dev/block/mtdblock2 /system
    /tmp/busybox mount -t yaffs2 /dev/block/mtdblock3 /data
    if (! df /dev/block/mtdblock2 | grep "256000") || (! df /dev/block/mtdblock3 | grep "711680"); then
    # Our partition sizes are not correct
    
    # unmount, format and mount system
    /tmp/busybox umount -l /system
    /tmp/erase_image system
    
    # unmount and format cache
    /tmp/busybox umount -l /cache
    /tmp/erase_image cache
    
    # unmount and format datadata
    /tmp/busybox umount -l /data
    /tmp/erase_image userdata
    
    # make sure sdcard is mounted
    if ! /tmp/busybox grep -q /mnt/sdcard /proc/mounts ; then
    /tmp/busybox mkdir -p /mnt/sdcard
    /tmp/busybox umount -l /dev/block/mmcblk0p1
    if ! /tmp/busybox mount -t vfat /dev/block/mmcblk0p1 /mnt/sdcard ; then
    /tmp/busybox echo "Cannot mount sdcard."
    exit 1
    fi
    fi
    
    # remove old log
    rm -rf /mnt/sdcard/cyanogenmod_bml.log
    
    # everything is logged into /sdcard/cyanogenmod.log
    exec >> /mnt/sdcard/cyanogenmod_bml.log 2>&1
    
    # write the package path to sdcard cyanogenmod.cfg
    if /tmp/busybox test -n "$UPDATE_PACKAGE" ; then
    PACKAGE_LOCATION=${UPDATE_PACKAGE#/mnt}
    /tmp/busybox echo "$PACKAGE_LOCATION" > /mnt/sdcard/cyanogenmod.cfg
    fi
    
    # Scorch any ROM Manager settings to require the user to reflash recovery
    /tmp/busybox rm -f /mnt/sdcard/clockworkmod/.settings
    
    # write new kernel to boot partition
    /tmp/flash_image boot /tmp/boot.img
    if [ "$?" != "0" ] ; then
    exit 3
    fi
    /tmp/busybox sync
    
    /sbin/reboot now
    exit 0
    fi

    Taking that a step further, each ROM maker could choose whether to check the sizes of MTD partitions and if in tolerance (some predefined range of acceptable) it would flash ROM package on existing partition sizes, OR if not acceptable it could wipe out the partition (and all data on it) and start over with preferred sizes. Right?

    Exactly.

    ---------- Post added at 12:14 PM ---------- Previous post was at 12:04 PM ----------

    Let me add one very important note. After flashing those different kernels and mounting partitions with them, when I finally booted my phone, it booted fine, but my home screen layout has reverted to stock layout, which suggests that some of my data was munched when I mounted /data using different sizes. My downloaded apps still show up in my launcher, so some of my data is still there, but obviously some of it was lost.

    I think it's safe to say that if partition layouts are changed, partitions will need to be erased and recreated.
    3
    If I install the new CM 7.2 RC0 MTD on my phone, then is there any way I can still dual-boot a TW rom from my SD card?
    Yes, although I don't think anyone's actually tested it. You'll need a modified initramfs for the TW kernel to mount partitions from the SD card, but otherwise any kernel should work.

    I prepared an EI22 almost-stock kernel (as in, bug-compatible with stock) for Warren and he posted some notes on his blog (see "Multiboot Android for Debugging/Testing" section) for getting a TW ROM on SD. The patch archive has a README with some better instructions, although I threw the whole thing together rather quickly.

    The tricky part is you'll have to flash back to EI22 first and make tar of /system (a .tar system dump, not an Odin-flashable .tar). Otherwise it works essentially same as the CM7 SD ROM I posted for the 1022 version a while back.

    I wonder why there's no kexec support in the MTD kernel.
    The EpicCM (now CM7 victory) kernel was forked from official GB sources very early on before the kexec patches were well-tested, and they just haven't been folded back in. That can be done, folks have just been very busy getting CM7 out the door.

    I'm guessing other MTD kernels are based off the CM7 one and so are missing the patches for that reason. There's nothing inherent about MTD that precludes the ability to patch in kexec support.

    but kexecing a boot.img is rather different than a zImage.
    I was looking into this last night. You should be able to kexec a boot.img as long as you extract boot.cpio.gz first and pass it into kexec with the --initrd parameter.

    However, I don't recommend using unmodified boot.img kernels as they contain stage1 in their embedded initramfs which, while shouldn't be executed, it's never removed either, and so takes up extra memory. For whichever boot.img kernel you're using, I recommend rebuilding it from source with CONFIG_INITRAMFS_SOURCE="" to force it to build with a stub embedded initramfs, then use boot.cpio.gz or recovery.cpio.gz with --initrd to avoid stage1 entirely.

    If you can't rebuild the boot.img kernel from source, you can repack it with a stub initramfs, but that's almost always harder to do.
    3
    Seems like a risky approach to map the partitions in the kernel and not have it fixed elsewhere, but I assume this is best solution for employing MTD. Wonder if any other approaches are available or were considered?
    There is an on-flash partition table, bml2 (not mapped in mtd), and it would certainly make sense to support it under MTD kernels to allow for different partition sizes. It's not done simply becuase there's no MTD-pathway code for reading the on-flash partition table. Partition sizes are hardcoded in the Samsung MTD driver on the Nexus S, and, since that's how the driver was written, was kept for Galaxy S MTD kernels.

    It's possible to add on-flash partition table support for MTD, but would require some amount of code. I'd guess it's a reasonably-sized project for someone sufficiently ambitious. The format is known for the relevant portion of the partition table, plus there's code in the tfsr driver for reading it also--which I wouldn't copy, but might help in making sense of it.

    Honestly there is NO point to using MTD/Yaffs2 if we don't make use of the ability to change partition sizes.
    To be honest, the main benefit of MTD vs BML is maintenance of the CM port. Since all other supported SGS devices use MTD, for the Epic to use MTD reduces the amount of our device-specific configutation. While a BML-based CM7 port is certainly doable (an has been done), being the sole BML device means there's more configuration junk that we have to maintain.

    The other major benefit of MTD would be the ability to forward port victory (Epic) support to a Nexus S-derived (S5PV210) ICS kernel, i.e., for CM9. With Samsung's proprietary storage stack, we're stuck on our present kernel version unless newer modules leak out somewhere. Given that Samsung appears uninterested in porting ICS to the SGS, those might be hard to come by, especially timely.

    Personally I don't think MTD is particularly advantageous to TW kernels, aside from where basing on the EpicCM kernel makes such kernels easier to maintain.