[Dev] Research on native ext4 partition

---

* Update * - 02/28/2012

I had a little chat with one of Archos dev, he told me that mmcblk0p3 was his gift for us!

So, what does it mean:
1) Archos is very developer friendly. We've known that already, but having them leaving us such entries in their tablet is very nice to see!
2) He told me that mmcblk0p3 isn't and won't be used by Archos firmware, we're completely free to use it like we want with SDE
3) With this partition, we can do very nice things, see below


Here is what can be done:
1) delete p3 and p4
2) recreate p3 as extended with any size we want
3) recreate p4 with what remains.
4) on extended p3, we can create p5, p6, ...

I did that on mine, I now have a 8 MB p5 for cwmod, a 380M p6 for rootfs+system, without any interference with stock firmware!

If someone wants to experiment with that, few advices:
- Be careful when you re-create p3 and p4, there is an empty space that should remain empty in the partition table (it contains bootloader). So when you recreate p3, you have to use the same start sector as before, not the one proposed by default.
- When you re-create p4, make sure that you use end of p3 + 1 as start for the same reason.
- Don't do fdisk if you have partition mounted, it would confuse kernel and could be dangerous. You need a specific initramfs to be able to do that.
- This is still very dangerous, be warned! invalid partition table and your tablet is toaster. Only change it if you know exactly what you're doing

As a sample, here is my current partition table:

Disk /dev/block/mmcblk0: 7457 MB, 7457472512 bytes
4 heads, 16 sectors/track, 227584 cylinders
Units = cylinders of 64 * 512 = 32768 bytes

              Device Boot      Start         End      Blocks  Id System
/dev/block/mmcblk0p1               5        1605       51200+ 83 Linux
/dev/block/mmcblk0p2            1605       10760      292969+ 83 Linux
/dev/block/mmcblk0p3           10760       22967      390625+  5 Extended
/dev/block/mmcblk0p4           22967      227584     6547764  83 Linux
/dev/block/mmcblk0p5           10760       11004        7813  83 Linux
/dev/block/mmcblk0p6           11004       22967      382796+ 83 Linux

------------
Previous text:

Hello All,

As I saw some interest in running firmware directly from ext4 partition, I'd like to share what I found out while I was toying with my tablet:

Having ext4 on gen9 is not as easy as gen8: gen8 has two virtual mmcs on top of a single physical mmc with a split managed at kernel level. So what was done was to reduce storage partition to add ext4 system and/or data partition. More, on gen8, SDE exports storage mmc as a disk to the host allowing resizing partitions in linux for instance.

We can't do that on Gen9: Gen9 has a single physical mmc, part 1 is rawfs, part 2 is system and too short to have unpacked system, part 3 is a very small unused small one, part 4 is data.

On top of part 4, there is an Archos specific trick that remaps /data/media to /storage, explaining why we don't have the same limit on data partition and that /data and /storage are sharing same space. And SDE doesn't export disk.

BIG WARNING BEFORE TRYING ANYTHING: DON'T TOUCH PARTITION 1, NEVER. If partition 1 is trashed or damaged, device is bricked definitively.

So, what is possible is to trash part 2, 3, 4 and re-create a different layout or trash part 4, create two or three extended partitions instead.

If you go option 1, it would have to be done completely from an initrd as you would kill the os you're running on . Option two could be done from adb if Android is properly stopped, but I didn't test it and don't know if kernel and/or Android are behaving well with extended partitions.

Personally, I did it a third way, I modified archos recovery: recovery has a special code that repartition disk if things goes very bad. I changed partition layout builded there, then trashed my partitions and go to recovery to have it re-created. This method has advantages, the partition layout is permanent, meaning that if for some reason I trash one partition it will be re-created with the right size.

However, it has many disadvantages: You have to be able to flash rawfs to be able to flash a recovery, and it's a bit hectic and dangerous to force the partition re-creation. That's why I didn't share it and probably won't as you really need to know what you're doing if you don't want a big brick.

Last thing, I didn't see that much improvement in speed compared to other methods like having a loopback on internal storage, not sure it's really worth the effort and the risks. I did it because I don't like loopbacks, but frankly, having it done safely is much efforts.

Good post.

Thanks for the information

it is great

Hi letama,

interesting thread again

Originally Posted by letama

Having ext4 on gen9 is not as easy as gen8: gen8 has two virtual mmcs on top of a single physical mmc with a split managed at kernel level. So what was done was to reduce storage partition to add ext4 system and/or data partition. More, on gen8, SDE exports storage mmc as a disk to the host allowing resizing partitions in linux for instance.

Nothing to add...
I found out that a second mbr is placed at the beginning of the "virtual" mmc block device 1. So the internal eMMC chip of the Gen8 devices additionally requires a special layout.

Originally Posted by letama

We can't do that on Gen9: Gen9 has a single physical mmc, part 1 is rawfs, part 2 is system and too short to have unpacked system, part 3 is a very small unused small one, part 4 is data.

Going down to hardware level, it seems that Archos went back to use raw NAND flashes instead of eMMC for the Gen9 devices.
I found a FCC document of the Archos 80 G9 showing one 8GByte NAND flash device placed on the mainboard.
Could be a prototype as well...

Additionally it could be possible, that the mtd block layout is already handled inside bootcode.
Anyway, what does a cat /proc/partitions print out?

Regards,

scholbert

/proc/partitions:

major minor  #blocks  name

   7        0     159412 loop0
 179        0    7282688 mmcblk0
 179        1      51200 mmcblk0p1
 179        2     195359 mmcblk0p2
 179        3       7840 mmcblk0p3
 179        4    7028160 mmcblk0p4

Hi Scholbert,

Originally Posted by scholbert

interesting thread again

Thanks!

Originally Posted by scholbert

Nothing to add...
I found out that a second mbr is placed at the beginning of the "virtual" mmc block device 1. So the internal eMMC chip of the Gen8 devices additionally requires a special layout.

Well, kind of, it's not really a special layout. From userspace linux point of view, the two virtual mmcs are regular disks, they could be formatted, partitioned, etc.

The mmc driver is doing all the "magic" (mmc_block.split kernel command line parameter):
- for virtual mmc0, virtual sector is mapped to physical sector 0 - split
- for virtual mmc1, virtual sector is mapped to physical sector split - end.

So, the second mbr is coming from a regular disk initialization, nothing special.

Originally Posted by scholbert

Going down to hardware level, it seems that Archos went back to use raw NAND flashes instead of eMMC for the Gen9 devices.

We're still eMMC afaik.

Originally Posted by scholbert

Additionally it could be possible, that the mtd block layout is already handled inside bootcode.

No, gen9 has regular partition scheme, I was able to repartition it without problem... The only difficulty is to do it without killing the device by accident as if you want to keep the 4 partitions layout, you have to kill system and data (and data means also internal storage) partitions, be sure that you don't kill rawfs partition in the process then rebuild them.

Nothing very complicated per se, I would do a ram only initrd with backup of system/data/storage via adb or on microsd. However, as microsd slot is not completely stable for me, the fullly automated option is a bit risky.

My /proc/partitions is this now

major minor  #blocks  name

 179        0    7282688 mmcblk0
 179        1      51200 mmcblk0p1
 179        2     439487 mmcblk0p2
 179        3       7840 mmcblk0p3
 179        4    6784032 mmcblk0p4

Hi Letama,

I see you have modify your part scheme but mmcblk0p2 is your android root part (like loop img) ?

You have rewrite your bootloader ? or just android kernel/initramfs (not SDE kernel/initramfs) ?

Have a nice day !

Thaolia

Hi Tha0lia,

Yes, my p2 is now rootfs+system, it's the content of the img, but directly on the partition, formatted as ext4.

No, it's not the bootloader, bootloader doesn't know anything about partition layout.

Like I explained, I did it by patching the recovery as it will recreate partitions if it detects if a partition is missing. Then, I deleted my system and data partition, rebooted in recovery to have it recreate the whole disk. Last was to modify SDE/Main kernel initrd to mount this partition instead of the loopback device.

You can't do this if you don't have a tool to flash recovery/main kernel. I wrote one, but as it's very dangerous for the tablet, I'm reluctant to let it out.

Back on the subject, this is something that could be done with sde only with a properly written initrd. The only downside is that if for some reason recovery detects a partition problem, it will revert to the old layout, but it should work. The other downside is that "stock boot" on main kernel will only work if you dedicate enough space to hold both the squashfs and the rootfs files. Mine doesn't have too as I modified also stock boot, but again, to do that, you need to flash rawfs init file and special flasher.

I don't have enough time (and energy, enough formats these days ) to do the SDE initrd right now, we'll see if someone takes the baby and if it's really worth it.

Woaw ! Good work ! Thanks for the complementary informations.

So my hack ("abcbox reboot_into -s sde" at boot) is more secure than flash rawfs if i have understood well.

Last question (sry ^^"), but do you have made a serial connection with your tablet for debugging or dev your rawfs flasher ?

Originally Posted by letama

Well, kind of, it's not really a special layout. From userspace linux point of view, the two virtual mmcs are regular disks, they could be formatted, partitioned, etc.

The mmc driver is doing all the "magic" (mmc_block.split kernel command line parameter):
- for virtual mmc0, virtual sector is mapped to physical sector 0 - split
- for virtual mmc1, virtual sector is mapped to physical sector split - end.

So, the second mbr is coming from a regular disk initialization, nothing special.

Yeah sure, i just wanted to point at that mbr...

Originally Posted by letama

We're still eMMC afaik.

Considering the device names it seems eMMC is used, makes sense as this is newer/better technology.
Maybe the FCC doc is of interested.
See attachment... on this device a NAND flash (H27UCG8UDA) is used.
This is definitely raw NAND.
So maybe this is a pre-series model...

Originally Posted by letama

No, gen9 has regular partition scheme, I was able to repartition it without problem... The only difficulty is to do it without killing the device by accident as if you want to keep the 4 partitions layout, you have to kill system and data (and data means also internal storage) partitions, be sure that you don't kill rawfs partition in the process then rebuild them.

Nothing very complicated per se, I would do a ram only initrd with backup of system/data/storage via adb or on microsd. However, as microsd slot is not completely stable for me, the fullly automated option is a bit risky.

Your power supply should be fail-safe at this point

Originally Posted by letama

My /proc/partitions is this now

major minor  #blocks  name

 179        0    7282688 mmcblk0
 179        1      51200 mmcblk0p1
 179        2     439487 mmcblk0p2
 179        3       7840 mmcblk0p3
 179        4    6784032 mmcblk0p4

Yeah cool stuff... still on my Gen8 though.

Keep on hackin'

scholbert