[WIP][ThinkTank] Alternate Boot Methods (Safely boot CM10 from usb)

[Trooper_Max]

This thread is for brainstorming and developing safe ways of booting our OUYAs to use custom ROMs.

I've developed an initial method that allows OUYA to boot CM 10 from a usb thumb drive that does not require you to mess with any of your internal storage at all! This allows you to keep your OUYA in pristine stock condition, while still enjoying CM 10 from an external thumb drive.

Unfortunately it does require you to setup the thumb drive in a particular way to allow it to be used and this may be difficult for some users without Linux experience. I hope some will find it useful and plan to develop easier methods in the future (probably involving a custom recovery image with the ability to setup the thumb drive for you)

I plan to post more details on how I achieved this so that others can use the knowledge and apply it to other roms or develop improved solutions, but I might work on some other things first. Mainly it involved unpacking/repacking the Android boot.img (google it and there are tutorials about how to do this, I also recommend checking out the "abootimg" program from the Ubuntu packages) and modifying a couple of the init/fstab files.

Note that the zip I am going to link to is not a flashable zip, extract it and follow the instructions in the README which I am including below. Also note the the Google apps are not included - so if you want them you will need to add them yourself. Just be aware things you add to the system partition will need correct permission set.

One of my other ideas I'm going to eventually try is swapping the boot/recovery partitions so that the device normally boots up into recovery and then will have an option to reboot into the recovery partition which would actually boot a full rom for regular use. This may still be dangerous on OUYA though, so is not recommended unless you know what you're doing. I have a Notion Ink Adam which I can boot into APX mode (a low-level nVidia recovery mode that can be used to restore the device in the case of bricking, unfortunately we do not have this level of acces on OUYA), so it will be much safer for me to experiment on it. I haven't really developed on it before though (in fact this OUYA bit is my first major Android development apart from doing/tweaking some CM builds from source myself for some of my devices), so it may take me some time to set stuff up to experiment with it.

Eventually I think we need some form of bootloader for the OUYA. I have read about some very interesting kexec hardboot patches that were developed for the original Nexus 7 that it would be very awesome if we could port to OUYA - that would allow us to boot the patched kernel and have it boot us into kernels/roms stored elsewhere.

Please post any feedback or ideas. Sorry the current method isn't that easy yet and I hope the instructions make sense - hopefully others can also help clarify.

If you'd like to donate to me, I'm certainly not going to turn you away, but keep the other devs in mind 'cause I haven't done that much yet! ( PayPal: [email protected] )

~Troop

 

[Trooper_Max]

ouya-sda123-boot by Trooper_Max

Safely boot your OUYA to CM 10 without messing with your internal storage! (fast thumb drive recommended!)

========
Credits:
========
khanning88 for the initial CM10 Experimental ROM - this is just a repackaging/reconfiguration of it.

mybook4, sonofskywalker3, rayman, professorpoptart for their CWM recovery
And of course the CyanogenMod/ClockworkMod team for the basis of everything!

=========
Contents:
=========
ouya-cm10-system.img:
ext4 system image dumped from CM 10 experimental OUYA rom

ouya-sda123-boot.img:
Android boot.img to boot CM 10 from thumb drive partitions (details below)

==============
Prerequisites:
==============
For this current method, you will need to be able to partition the thumb drive into three ext4 partitions. This probably means you need Linux - if you don't have a Linux system, I highly recommend checking out pendrivelinux.com for methods of booting Linux off of a thumb drive (I recommend Ubuntu 12.04 LTS or whatever version you are comfortable with).

I hope to develop more methods in the future that will be easier than this, but this is the initial method.

=============
Instructions:
=============

1. Partition the thumb drive with three ext4 partitions - note that they must be the first three partitions on the thumb drive, and I would likely just dedicate a thumb drive to this. I recommend using Disk Utility or gparted (usually both available from an Ubuntu thumb drive).
* The first partition will be the data partition and should be the largest
* Note that Android will create the virtual sd card at /data/media
* The second partition will be system (OUYA internal system is 512 M)
* The third partition will be cache (OUYA internal cache is 768 M)
* Feel free to adjust the size of the partitions, but I'd recommend sticking close to the stock sizes
* I've tested this on an 8 GB as well as a smaller 4 GB thumb drive, bigger should not be a problem

2. Note the device name of your thumb drive - It will likely be sdX where X is a letter corresonding to the order it was mounted in - I would expect it to be sdd if it happens to be the fourth drive connected to your machine. The system partition will then be sdX2.

3. Write the ouya-cm10-system.img to the second partition of the thumb drive. It is crucial that the files get copied into the partition with the correct permissions.
* The easiest way to ensure this is to use dd to do a byte for byte copy of the system image directly over the partition, but this is also very dangerous if you type it wrong, so be sure you have the write device name for your thumbdrive.
* I recommend looking in the Disk Utility or gparted, or running "mount", "df -h", and "cat /proc/partitions", to make sure you have an understanding of what drives are what device names before continuing
* Once you are certain of the device name, ie sdX2, where X is the letter for the thumb drive and 2 denotes the second partition which we are using as system, run "dd if=ouya-cm10-system.img of=/dev/sdX2 bs=4M" as administrator (on Ubuntu either by sticking the word "sudo" in front of the command or running "sudo su" first to switch to root)
* This command will not show any output until the end and may take a little while.

4. Clean/fix the filesystem and resize the filesystem to fill the partition.
* Run "esfsck -fp /dev/sdX2" as administrator
* Run "resize2fs /dev/sdX2" as administrator

5. Thumb drive is now ready (the data/cache partitions can be empty ext4, Android will fill them in). Connect it to the OUYA. Through a hub is fine - just be sure it is the only thumb drive connected at boot.

6. Boot the OUYA into fastboot mode. The only real way to do this right now is to first boot up the OUYA normally, then use "adb reboot bootloader" to reboot into the bootloader. You should be able to run "fastboot devices" then and see a device listed.

7. Boot ouya-sda123-boot.img using the command "fastboot boot ouya-sda123-boot.img". You should see it download to the OUYA and it should start booting.

8. Wait patiently. Remember that CM boots slow the first time and depending on the speed of your thumb drive may boot even slower. You can however type "adb devices" to see if it has started the adb daemon. If you don't see a boot animation after a while, you can try running "adb shell" and if you get a permission error, it probably means you didn't flash the system partition correctly, but hopefully all goes well. Note though that once it gets farther into the boot sequence it will turn off the adb daemon so you will lose adb access until it boots up and you can re-enable it.

9. Enjoy CM 10 on your OUYA without having messed with your internal storage! Just be sure not to let your OUYA fall asleep, as it may not be able to wake back up! I'm guessing this is because I was using a hub and so when it falls asleep, the thumb drive essentially gets disconnected and it cannot immediately find it again when it tries to resume. I'd recommend using a wakelock application or power toggle to keep the screen on all the time, etc.

========
For Help
========

Look for us on the XDA Developers Forum under the appropriate threads! Keep in mind that using this method to boot CM 10 may introduce new bugs that would not have occurred using CM 10 the regular way, so be sure to report problems in the appropriate place and mention what methods you used!

http://troopermax.com/releases/ouya-sda123-boot.zip
(be gentle - it's 145 MB, I use shared hosting, so feel free to mirror)
md5sum: 49c8e16e27b6deb9d1e8e86363b56f2f

Mirror: http://www.mediafire.com/download/hban76kzeys6ybd/ouya-sda123-boot.zip

~Troop

 

[Trooper_Max]

I'm including some rough developer details here for now about how I did it.

Many thanks once again to the OUYA CWM recovery, I found it insanely useful: http://forum.xda-developers.com/showthread.php?t=2295645

The main part was unpacking/modifying/repacking the boot.img (which contains the kernel/ramdisk/etc)

abootimg helped with this greatly:
http://manpages.ubuntu.com/manpages/precise/man1/abootimg.1.html

Then I followed instructions I found via google for unpacking/repacking the ramdisk:
http://android-dls.com/wiki/index.php?title=HOWTO:_Unpack,_Edit,_and_Re-Pack_Boot_Images

The partitions the OUYA uses are mostly defined in fstab.cardhu:

# Android fstab file.
#<src> <mnt_point> <type> <mnt_flags> <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

# We remount because we first mount as rw in order to generate NVSI symlink. See init.rc for details.
/dev/block/platform/sdhci-tegra.3/by-name/APP /system ext4 ro,remount wait
/dev/block/platform/sdhci-tegra.3/by-name/CAC /cache ext4 noatime,nosuid,nodev,journal_async_commit,data=writeback,nodelalloc,errors=panic wait
/dev/block/platform/sdhci-tegra.3/by-name/UDA /data ext4 noatime,nosuid,nodev,journal_async_commit,data=writeback,nodelalloc,errors=panic wait,encryptable=/dev/block/platform/sdhci-tegra.3/by-name/MDA

I found that I could use fastboot to boot into the CWM recovery and then use adb pull to dump images of the internal OUYA partitions, using the paths above, ex. "adb pull /dev/block/platform/sdhci-tegra.3/by-name/APP ouya-system.img" to dump the system partition. I did a regular clockwordmod backup first, but adb pull was useful here for pulling raw images.

So after backing up my OUYA stock system, I actually did flash the CM10 rom so that I could dump the partitions to my computer using adb. The main reason I did this was to ensure the system partition gets generated with the correct permissions. I didn't boot into CM10 though, though I was tempted. Instead I restored back to my backup to put my OUYA back in stock condition.

With the system image dumped into a file, I used resize2fs to shrink the file to the minimum size (resize2fs -fM ouya-system.img) This probably wasn't absolutely necessary since it would compress down in the zip, but this allows it to be written to smaller partition sizes.
http://manpages.ubuntu.com/manpages/precise/en/man8/resize2fs.8.html

Here is what I modified the fstab.cardhu to:

# Android fstab file.
#<src> <mnt_point> <type> <mnt_flags> <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

# We remount because we first mount as rw in order to generate NVSI symlink. See init.rc for details.
/dev/block/sda2 /system ext4 rw wait
/dev/block/sda3 /cache ext4 noatime,nosuid,nodev,journal_async_commit,data=writeback,nodelalloc,errors=panic wait
/dev/block/sda1 /data ext4 noatime,nosuid,nodev,journal_async_commit,data=writeback,nodelalloc,errors=panic wait

Pretty straightforward, except that I removed the encryptable part on /data (not sure how that bits work and if I might have broken it) and made system read/write.

Also note, that I removed the initial read/write mounting of system in init.cardhu.rc since I was having trouble with it mounting from there (I'm guessing because my usb hub/thumb drive weren't yet available when that tried)

Here's the relevant part of init.cardhu.rc:
Before:

on fs
setprop ro.crypto.tmpfs_options size=128m,mode=0771,uid=1000,gid=1000
setprop ro.crypto.umount_sd false
# PLEASE DO NOT REMOVE NVSI SYMLINK! IF CHANGES ARE NEEDED PLEASE ENSURE THAT NVSI SYMLINK IS ALWAYS CREATED.
# Mount system to allow NVSI symlink
mount ext4 /dev/block/platform/sdhci-tegra.3/by-name/APP /system rw
# Create NVSI filter symlink
symlink /data/data/com.nvidia.NvCPLSvc/files/com.nvidia.nvsiutilv1.xml /system/etc/permissions/com.nvidia.nvsiutilv1.xml
mount_all /fstab.cardhu

#chmod for OUYA parameters
chmod 0644 /dev/block/mmcblk0p5

After:

on fs
setprop ro.crypto.tmpfs_options size=128m,mode=0771,uid=1000,gid=1000
setprop ro.crypto.umount_sd false
# PLEASE DO NOT REMOVE NVSI SYMLINK! IF CHANGES ARE NEEDED PLEASE ENSURE THAT NVSI SYMLINK IS ALWAYS CREATED.
# Mount system to allow NVSI symlink
#mount ext4 /dev/block/sda2 /system rw
mount_all /fstab.cardhu
# Create NVSI filter symlink
symlink /data/data/com.nvidia.NvCPLSvc/files/com.nvidia.nvsiutilv1.xml /system/etc/permissions/com.nvidia.nvsiutilv1.xml

#chmod for OUYA parameters
chmod 0644 /dev/block/mmcblk0p5

Basically I commented out the mount command - before it was mounting system read/write for a moment to insert the symlink and let the filesystem update itself before remounting read-only in the fstab. The way I do it now it is simply mounted read/write from the fstab and left that way (some may consider this unsafe, but it is good for development... not too hard to remount it read-only if you so desire) I also shifted the symlink command down below the mount all command so it would hopefully still work... not sure what that was for, but they made it sound really important! XD

I also commented out this part about the usb drive since it doesn't really make sense anymore:

# Mount usb drives as /usbdrive. Generally usb drives are formatted with FAT
# filesystem, so we support FAT as of now.
#on device-added-/sys/block/sda
# mount vfat /dev/block/sda /mnt/usbdrive

I think those were all the changes I made before repacking it, though it did take me 5 attempts to get a working ramdisk so I may have lost track of something at some point. Those should be the key changes at least.

I also initially used raw dumps of the data and cache partitions on my thumb drive as well to make sure it would work. Then I tested to make sure it would work if they were empty ext4 partitions, which they do - Android fills them in as necessary. So only the system image was necessary to include. (and of course the boot.img for fastbooting)

This was a little bit harder than I had anticipated because I'd previously only looked into this type of thing for Gingerbread devices which aren't set up the same way. I'd kinda been hoping to get a solution where the user could just drop a system.img and data.img onto a fat32 usb thumb drive and the boot.img would loop mount them off the thumb drive. I haven't tried this for Gingerbread, but it seems pretty straightforward. It's not straightforward anymore here since fat32 sdcards aren't used anymore. Being familiar with Linux though, I am happy they are using ext4 everywhere and using the same space alloation for /data and /sdcard so you don't have to worry about which one to make big, but it does make some things more complicated for more average users. It's probably still possible to get the OUYA to do what I wanted, but more complicated and not sure it's worth pursuing over other methods which may be more fruitful.

Hope that's enough details to enlighten some people who may be wondering about the methods used here. My goal is to not only develop new ways of booting the OUYA, but also to share how I accomplished things so that others can learn from it.

~Troop

 

[kairnage]

Is this similar to the Nook HD + CM10 boot?

---------- Post added at 05:07 AM ---------- Previous post was at 04:58 AM ----------

Wait that was blank a second ago. I am downloading now. I will mirro with my mediafir stuff if I can get it to work.

 

[Trooper_Max]

Is this similar to the Nook HD + CM10 boot?

---------- Post added at 05:07 AM ---------- Previous post was at 04:58 AM ----------

Wait that was blank a second ago. I am downloading now. I will mirro with my mediafir stuff if I can get it to work.

Sorry, I was setting up the posts...

I'm not sure this is similar to any solution currently used on other devices, but I do know that other developers have used similar methods to develop their roms.

I don't have a Nook and haven't looked into what they use too much, but we may very well be able to learn from their work.

~Troop

 

[Trooper_Max]

So I dug up an old nvflash package from my Notion Ink Adam and tested it to make sure it still works (with the Adam in APX mode it allows me to directly flash the device with a new partition layout and flash the partitions directly). After verifying I had it setup correctly, I swapped the boot and recovery images and ran it again and sure enough... now when I boot my Adam normally, the recovery rom comes up, and when I boot my Adam into recovery it loads the normal rom - so the theory seems to work.

So I think the next thing I will try is building a new recovery for OUYA from source and adding an option to reboot into "recovery" to the menu, allowing it to be stored on the boot partition and the real boot.img to be stored on the recovery partition, allowing you to boot the OUYA and it comes up in recovery, then you select the new reboot option and it would reboot into "recovery" which would load a full rom. You could then safely flash boot images to the recovery partition without fear of bricking your device.

I think it should work nicely, but not sure how long it will take me since I need to set my build machine back up.

For those who are confused, read more about how these ideas started below!

~Troop

I think this is a great idea that needs more attention:

Devs, I propose the following to get rid of the potential brick risk:

Since we can't get into recovery manually (via HW buttons), let's reverse the role of SOS (recovery) and LNX (kernel). Since LNX is the booting kernel partition, let's flash recovery there and flash the kernel to the recovery partition. I believe we could do this by modifying the fstab and having our updater-scripts flash to the appropriate partitions.

From a cold start, a user will enter recovery (a minor inconvenience for safety). Depending on how we modify the recovery.fstab, getting to the ROM could be as simple as pressing power twice (recovery does a reboot system now and its fstab has the system actually reboot to the recovery partition, which is the ROM's kernel).

Definitely not straightforward, but should prevent bricks. Thoughts?

Sent from my Nexus 7 using xda premium

But then I'm definitely biased, because I had the same idea:

I agree with everyone it would be nice if we had a hardware means to boot into some kind of recovery or flashing mode to flash stock images.

But I was trying to think if there is anything we can do if we don't get that... (guess that makes me a pessimist >_>)

Let me preface this with don't try anything I'm describing here unless you know what you're doing, these are half-baked thoughts of someone who doesn't even have their OUYA yet:

What if we could swap the boot and recovery partitions? (I haven't received my OUYA yet, so I'm not even aware of what partitions it has, so assuming it has them, and yeah, it wouldn't be quite that simple)

Basically what I mean is when the device tries to boot normally, it would boot up a stable recovery rom. (ie the boot partition that normally has the kernel and then loads the rest of the rom instead just has a recovery rom)

Then you have the recovery partition be what boots the full rom. That way you only chance bricking your device when you flash the boot partition, which if we can get a stable recovery rom that works for this, wouldn't be often. We would probably want to modify the recovery to have a 3 second/configurable timer where if you don't do anything it boots into the "recovery" partition which would boot up the full rom.

Basically the boot partition becomes a new recovery rom which gets used like GRUB to boot into the "recovery" partition which boots up your actual rom, or maybe it could also boot from USB or netboot or whatever...

Pretty much what we need is a solid bootloader, sound about right? Let me know if any of this makes sense/doable or if I'm entirely off base here! I don't know if I will have the time to try any of this when I do get my OUYA, but wanted to share some ideas, please proceed at your own risk!

EDIT: Alternatively, the boot partition could be left stock and the recovery partition could be used as a bootloader to boot into USB or other options for loading roms without messing with the stock boot experience and risking bricking the device. ie in this configuration if you boot your device normally it would be stock, if you reboot into "recovery" it would load up a custom rom. Or instead of a custom rom, a custom recovery with bootloader capabilities.

~Troop

And I do plan to try this idea out eventually, but I've got some other ideas I intend to try out first.

The first one I hope to have done by the end of this weekend (hopefully sooner though) is to reconfigure this experimental CM build to be usable without touching any of the internal storage space. I plan to have a modified boot.img that will load system/data/sdcard folders off a usb thumb drive. This method would be completely safe, though a little inconvenient because you would need to use adb to reboot the OUYA into bootloader mode so that you can then fastboot it to load the modified boot.img, but after that it could be disconnected and would be running using only external storage. You're essentially using a computer to jumpstart your OUYA into CM10, while leaving everything on the OUYA itself in pristine stock condition. Then I plan to try out arm versions of adb/fastboot so I can use my tablet or possibly phone (USB OTG) to jumpstart the OUYA instead of having to rely on my computer all the time.

I think this will be an extremely safe way of using external ROMs until we can get some kind of special bootloader figured out.

So if anyone is thinking about trying CM10, but leery of messing with their OUYA, just wait another week or so and I should have a safe, non-intrusive solution worked out!

I welcome any input/thoughts on these ideas! And if anyone knows how to do the things I'm describing and wants to beat me to the punch, feel free and run with these ideas - I won't mind as long as you share your work and give me a little credit if any credit is due!

~Troop

 

[kairnage]

I just ordered a 64GB PNY off Amazon. Once I get it Tuesday I am going to try this. I am uploading to my mediafire now to mirror.

 

[Trooper_Max]

So I dug up an old nvflash package from my Notion Ink Adam and tested it to make sure it still works (with the Adam in APX mode it allows me to directly flash the device with a new partition layout and flash the partitions directly). After verifying I had it setup correctly, I swapped the boot and recovery images and ran it again and sure enough... now when I boot my Adam normally, the recovery rom comes up, and when I boot my Adam into recovery it loads the normal rom - so the theory seems to work.

After some more playing with my Adam, I noticed that when I reboot into "recovery" which I have booting a CyanogenMod rom and then I reboot and from the reboot menu select "Reboot", it reboots into CyanogenMod again (even though I didn't expect it to). From the reboot menu I tell it to reboot into "recovery" and it boots into CM again, which that part makes sense, since I have the CM boot image stored in the recovery partition. So I got to thinking that maybe when you tell it to reboot into recovery it sets an SOS signal (the recovery partition is sometimes refered to as SOS) and when you boot a normal rom from recovery, it never clears that signal because it isn't normally booting that way. Whereas a true recovery rom knows it needs to clear that signal so you don't get stuck booting into the recovery rom all the time.

I was close - I scanned the recovery source on CM's github and found this bit:

// Reset to normal system boot so recovery won't cycle indefinitely.
struct bootloader_message boot;
memset(&boot, 0, sizeof(boot));
set_bootloader_message(&boot);

So there seems to be a "bootloader message" that tells it whether to boot from the boot or recovery partition. I was going to say we need to be careful how we reboot the device and probably modify the custom rom to be able to reset the boot message, but I just realized we can do better and use this to our advantage. We could set our custom rom to always clear the boot message so that you have to go through the recovery rom each time to boot the system normally, but we could also have the option of selectively resetting the bootloader message - purposefully get ourselves stuck rebooting into "recovery" all the time because that is our normal rom, then from our normal rom have the option of resetting the bootloader message when we want to go into our recovery image stored on the boot partition.

Something to think about - I don't like getting stuck out of my recovery image either, so we gotta be sure we can reset the bootloader message when we need to... I'm going to play with this some more tomorrow, but not sure how far I will get.

EDIT: More info on the bootloader message - seems it is stored on the misc partition. The more I think about this too, the more I'm not sure swapping the boot/recovery is much safer - if we had them swapped, but then the rom gets stuck in a boot loop before it can reset the bootloader message, we'd be stuck always booting into "recovery" and into the boot loop, so we'd be in just as much trouble as if we'd gotten our boot partition to cause boot loops. So whereas normally flashing the boot partition is potentially dangerous, in this scenario flashing the recovery partition would be potentially dangerous - you gotta be sure it's going to be able to reset the bootloader message. We could try to mitigate this by modifying the boot process to reset the bootloader message very early in the boot process (ie before the potential bootloop) so that if we get stuck, the next time we reboot, we'll reboot normally into the recovery rom.

But then we could also do just the opposite of that without swapping the boot/recovery partitions - very early in the boot process, modify the bootloader message to tell the bootloader to boot into recovery - that way if we get stuck the next time we boot up will be into recovery. Then later when the system successfully boots, we could reset the bootloader message so that after a successful boot the next boot will be another normal one instead of into recovery. If we get this right a failed boot would automatically take us to recovery on the next boot, while a successful boot would boot normally on the next boot. So this would be a way to build in some "brick-protection" into the boot.img to make it safer to flash. It'd probably be safest to not reset the bootloader message until the user actually selects the shutdown option - that way any abnormal reboot would cause it to come up into recovery mode.

So I'm not sure swapping boot/recovery partitions really buys us anything anymore, since it also swaps which partition is potentially dangerous to flash. The solution in either case is going to be to cleverly manipulate the bootloader message as described above. So we might just want to not swap them to minimize confusion and focus instead on building "brick-protection" like this into our boot images.

/* Bootloader Message
*
* This structure describes the content of a block in flash
* that is used for recovery and the bootloader to talk to
* each other.
*
* The command field is updated by linux when it wants to
* reboot into recovery or to update radio or bootloader firmware.
* It is also updated by the bootloader when firmware update
* is complete (to boot into recovery for any final cleanup)
*
* The status field is written by the bootloader after the
* completion of an "update-radio" or "update-hboot" command.
*
* The recovery field is only written by linux and used
* for the system to send a message to recovery or the
* other way around.
*/
struct bootloader_message {
char command[32];
char status[32];
char recovery[1024];
};


/* Read and write the bootloader command from the "misc" partition.
* These return zero on success.
*/
int get_bootloader_message(struct bootloader_message *out);
int set_bootloader_message(const struct bootloader_message *in);

I just ordered a 64GB PNY off Amazon. Once I get it Tuesday I am going to try this. I am uploading to my mediafire now to mirror.

Sounds nice - I'm tempted to get a nice big new thumb drive myself... I don't have any that big >_>

~Troop

 

[dexter84]

I've been following your think tank and I agree with your latest idea 100%
Sending a message for bootloader to always boot into recovery if the system wasn't shut down normally is clean and simple.
I am curious though what about development, with nvflash requiring encryption of commands first mistake will render your ouya unusable.

 

[Trooper_Max]

I've been following your think tank and I agree with your latest idea 100%
Sending a message for bootloader to always boot into recovery if the system wasn't shut down normally is clean and simple.
I am curious though what about development, with nvflash requiring encryption of commands first mistake will render your ouya unusable.

Glad you agree with my new idea!

As far as development, yeah, that is why I used my Notion Ink Adam for some of the testing that I did.

As far as developing for OUYA, I don't think we are limited by not having nvflash, we just have to be more careful. Personally, if I were developing anything, I would use my above methods to fastboot boot images that load everything else from a thumb drive at least until whatever I'm developing becomes stable enough that I want to use it more permanently. Then I would be very careful about how I flash it.

Heck, if you wanted to be really safe, you could flash recovery images to both the boot and recovery partitions and just use external boot methods

In the long term though, I hope we can develop a bootloader based on the kexec hardboot patches for the original Nexus 7, which would allow booting completely off boot images stored in other locations, loading partitions from other locations. Other locations could even just be the virtual sd card if you don't want to use external media, but at that point we'd have a safe extra bootloader stored on the boot partition.

http://forum.xda-developers.com/showthread.php?t=2104706

Hope that makes sense.

~Troop

 

[dexter84]

I was not aware that any content of RAM can survive a reboot, hard kexec booting custom OS from USB or image is definitely a safe solution, looks like all the pieces are there.
Wish I had something more than ouya alone, since you have your Adam are you going to try to develop something ?

 

[Trooper_Max]

I was not aware that any content of RAM can survive a reboot, hard kexec booting custom OS from USB or image is definitely a safe solution, looks like all the pieces are there.
Wish I had something more than ouya alone, since you have your Adam are you going to try to develop something ?

I do intend to, but I don't have a great deal of time to devote to Android development, so not sure how long it will take me. I'm trying to find what time I can though. This past weekend/last week I had a bit more than usual.

~Troop

 

[kairnage]

http://troopermax.com/releases/ouya-sda123-boot.zip
(be gentle - it's 145 MB, I use shared hosting, so feel free to mirror)
md5sum: 49c8e16e27b6deb9d1e8e86363b56f2f

~Troop

Here is a mirror for you.

http://www.mediafire.com/download/hban76kzeys6ybd/ouya-sda123-boot.zip

I tried this on my new 64GB and it wouldn't boot, but it's a brand new card so it might be the problem. I'll try a known good card later.

 

[Trooper_Max]

Here is a mirror for you.

http://www.mediafire.com/download/hban76kzeys6ybd/ouya-sda123-boot.zip

I tried this on my new 64GB and it wouldn't boot, but it's a brand new card so it might be the problem. I'll try a known good card later.

Sorry you had trouble - a couple other things to be wary of:

Have only the one thumb drive connected, at least until it boots - the boot.img looks for the data partition as sda1, the system partition as sda2, and the cache partition as sda3. If some other storage device is connected, it might be labeled sda by the OUYA depending on the order it finds them in and then the thumb drive might become sdb or so on.

It worked for me through a hub, but it may also be worth trying it directly if you have trouble.

Guessing you did this, but give it some time, booting from thumb drive especially on the first boot is slower. It's not too long, but it was long enough that for a moment I thought I'd failed again. Of course the thumb drive I was using wasn't built for speed, so YMMV.

If you're still having trouble - even on my failed attempts at boot images for this (it took me 5 tries at getting the ramdisk to get it bootable), I was still able to connect to it using adb. I ran adb on Linux, running adb as root to bypass all that udev configuration stuff - it's just easier, and if the OUYA/device failed to mount the system partition, it might not match the configuration you expect anyway. If you're able to run "adb shell" but it gives you a permission error about not being able to access the shell binary on the system partition, that probably means you didn't setup the system partition correctly - it's crucial that it get set up with the right permissions, which is why I provided the image so it could just be written directly with dd.

I'm going to try to put together a recovery image that can partition the thumb drive for you and flash an arbitrary update.zip to the thumb drive, but not sure when I will be able to have that ready. The initial version will not be able to patch the boot.img though, so that will still need to be modified like the one I have provided, but maybe in a future version I can get it to also modify the boot.img for loading the information from the thumb drive.

Let me know if you continue to have issues or if you do get it working! And if anyone else has got it working, please post so I can at least know someone got it working. It worked for me, and I don't see why it shouldn't work for anyone else, but if there is something conflicting I want to figure it out.

EDIT: I just remembered when I was looking into the swapping boot/recovery stuff, I noticed in the recovery source that there already are some tools that might be able to help partition the thumb drive and fix permissions included in the recovery source:
https://github.com/CyanogenMod/android_bootable_recovery/tree/jellybean/utilities

Tonight I'll see if I can put together some new instructions using those tools that might be easier and less error-prone. I'm hoping those tools can be applied to the thumb drive anyway, I'll try and figure that out tonight.

~Troop

 

[kairnage]

Sorry you had trouble - a couple other things to be wary of:

Have only the one thumb drive connected, at least until it boots - the boot.img looks for the data partition as sda1, the system partition as sda2, and the cache partition as sda3. If some other storage device is connected, it might be labeled sda by the OUYA depending on the order it finds them in and then the thumb drive might become sdb or so on.

It worked for me through a hub, but it may also be worth trying it directly if you have trouble.

Guessing you did this, but give it some time, booting from thumb drive especially on the first boot is slower. It's not too long, but it was long enough that for a moment I thought I'd failed again. Of course the thumb drive I was using wasn't built for speed, so YMMV.

If you're still having trouble - even on my failed attempts at boot images for this (it took me 5 tries at getting the ramdisk to get it bootable), I was still able to connect to it using adb. I ran adb on Linux, running adb as root to bypass all that udev configuration stuff - it's just easier, and if the OUYA/device failed to mount the system partition, it might not match the configuration you expect anyway. If you're able to run "adb shell" but it gives you a permission error about not being able to access the shell binary on the system partition, that probably means you didn't setup the system partition correctly - it's crucial that it get set up with the right permissions, which is why I provided the image so it could just be written directly with dd.

I'm going to try to put together a recovery image that can partition the thumb drive for you and flash an arbitrary update.zip to the thumb drive, but not sure when I will be able to have that ready. The initial version will not be able to patch the boot.img though, so that will still need to be modified like the one I have provided, but maybe in a future version I can get it to also modify the boot.img for loading the information from the thumb drive.

Let me know if you continue to have issues or if you do get it working! And if anyone else has got it working, please post so I can at least know someone got it working. It worked for me, and I don't see why it shouldn't work for anyone else, but if there is something conflicting I want to figure it out.

EDIT: I just remembered when I was looking into the swapping boot/recovery stuff, I noticed in the recovery source that there already are some tools that might be able to help partition the thumb drive and fix permissions included in the recovery source:
https://github.com/CyanogenMod/android_bootable_recovery/tree/jellybean/utilities

Tonight I'll see if I can put together some new instructions using those tools that might be easier and less error-prone. I'm hoping those tools can be applied to the thumb drive anyway, I'll try and figure that out tonight.

~Troop

I could get into ADB as well but was only seeing the stock system. Actually now I think it was the adapter I was using. At first I assumed it was the microsd because it would hang on boot. But after trying a known good card it did the same. I remembered having issues with an adapter that has the indicator light to show it was connected on a tablet before. I tried a generic Kingston adapter and it sees the cards now. Of course I had already wiped them back to stock partitions. LOL Oh well, start again tomorrow.

 

[Trooper_Max]

For anyone who doen't have a Linux box handy or an extra thumb drive to boot a live USB, here are some quick instructions on how to partition the thumb drive from the OUYA itself. You can connect the thumb drive either before or after booting, it doesn't really matter. I'm using all fastboot/adb commands here so they are easy to recognize. If you're knowledgeable about how to use adb/fastboot, feel free to execute the commands however you're comfortable.

Type all the adb/fastboot commands as specified - if it doesn't begin with adb or fastboot, it isn't a command. All /dev paths you type into the commands should be related to /dev/block/sda (your thumb drive), not any internal partitions or you could mess up your OUYA!!!

Boot up the OUYA normally, then reboot into the bootloader:
adb reboot bootloader

Boot CWM using fastboot:
fastboot boot OuyaCWMrecovery6.0.3.2.img

Print partition information for the thumb drive:
adb shell parted /dev/block/sda print

Model: USB Flash Memory (scsi)
Disk /dev/block/sda: 8128MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 6000MB 6000MB primary ext4
2 6000MB 7000MB 1000MB primary ext4
3 7000MB 8000MB 1000MB primary ext4

(note in the example output above I already have it partitioned)

Here I was using an 8 GB thumb drive, you can see the size where it says 8128MB.

Delete Partitions from thumb drive: (be sure you don't have anything important on your thumb drive)
adb shell parted /dev/block/sda rm 1
adb shell parted /dev/block/sda rm 2
adb shell parted /dev/block/sda rm 3
(note that you only need to delete partitions that showed up in the table when you printed it, here I deleted 1,2,3, since I had 3 partitions above and the parted commands will say you may need to update your fstab, but that is not relevant here so don't worry about it)

Create 3 primary partitions (adjust the sizes as necessary):
adb shell parted /dev/block/sda mkpart primary 1 6000
adb shell parted /dev/block/sda mkpart primary 6000 7000
adb shell parted /dev/block/sda mkpart primary 7000 8128

Check your paritions again:
adb shell parted /dev/block/sda print

Format your partitions as ext4:
adb shell mke2fs -t ext4 /dev/block/sda1
adb shell mke2fs -t ext4 /dev/block/sda2
adb shell mke2fs -t ext4 /dev/block/sda3

Push the cm10 system image onto the device: (this will take a little while)
adb push ouya-cm10-system.img /tmp/ouya-cm10-system.img

Check that it copied correctly:
adb shell ls -l /tmp

-rw-rw-rw- 1 root root 260308992 Jul 27 2013 ouya-cm10-system.img
-rw-rw-rw- 1 root root 3587 Jan 1 02:12 recovery.log

(Note the size... I'm pretty sure copying it /tmp put it into the ramdisk, but our OUYA has enough ram for this)

Write the cm10 system image onto /dev/block/sda2: (this will take a little while)
adb shell dd if=/tmp/ouya-cm10-system.img of=/dev/block/sda2 bs=4M

Check/fix the filesystem:
adb shell e2fsck -fp /dev/block/sda2

Resize the filesystem to fill the partition:
adb shell resize2fs /dev/block/sda2
(Unfortunately, it looks like resize2fs is not included in CWM, so this step would need to be done on the computer)

At this point, the thumb drive should be good, so you just gotta reboot back to the bootloader and boot the boot image:

Reboot to the bootloader again:
adb reboot bootloader

Boot the modified boot image I provided:
fastboot boot ouya-sda123-boot.img

Not sure if that's necessarily easier or harder to follow than the other instructions... I'd probably rather do the partitioning on a desktop with GUIs, but for some who don't have a Linux box handy it might be easier to use the OUYA as a little Linux box

I'm sure you noticed this method still used dd to write the system image, just run from the OUYA instead. I looked into just extracting the system files from the CM10 zip onto the OUYA, but fixing the permissions was not as easy as I thought it would be - the /system partition permissions are actually fixed by the updater script as referred to here:
http://fokke.org/site/content/howto-create-android-updatezip-package
It's actually a scripting language and the script also creates symlinks and does other stuff.

You'd have to extract/modify the updater script to not use the internal partitions and then execute it (I think using the edify program)... still less straightforward than using dd if you have the image handy like I gave you... But I may look into this as another option. The knowledge will certainly be relevant when I look to build a custom CWM recovery to allow you to flash zips to the thumb drive instead of the internal partitions.

~Troop

 

[Trooper_Max]

Thanks to csonger for feedback, which may be handy if you have system partition issues:

I tried both methods (under linux and with ouya as well), but neither of them worked for me. My 8 gig pen couldn't boot.
The first thing is that the system image is 256MB and the tutorial says the System partition needs 512MB or 1GB.

After the boot failures I checked the pendrive. The second (system) partition was 256MB. It is ok when we use dd, but it was full. 100% usage.
First I run the e2fsck command to check for filesystem problems and it found some inode issues on it.
Following the fs fix the resize2fs command did the job and I had got a 1 gig System partition.

After that my pendrive could boot the cyanogenmod image without problems.

May be someone else has the same problem as me. Please share with them.

Thanks for your work:
csonger

I didn't have those issues myself for some reason, but I'll investigate and see what instructions I need to update!

Thanks again for the feedback!

~Troop

 

[kairnage]

Finally got my Linux partition back up and running so I can give this a go again.

 

[kairnage]

It's been trying to boot CM for about 15 minutes now, ADB does see it, but I think it might no like the partitions I did.I did 2GB for system and 1GB for cache. I will repartition and try again.

 

[Trooper_Max]

It's been trying to boot CM for about 15 minutes now, ADB does see it, but I think it might no like the partitions I did.I did 2GB for system and 1GB for cache. I will repartition and try again.

That should be fine, I would think... If you continue to have trouble, try running e2fsck and resize2fs on the system partition, as csonger suggested. (In case you didn't already)

I was confused when I came up with the instructions that it worked without me doing resize2fs since I figured that would be needed to expand the filesystem to fill the partition, but somehow it worked anyway and even looked like it was expanded, but I'd probably just forgot to refresh my view after dd.

Note that you can run the commands with just the partition as an argument: (I added -fp to e2fsck to automatically find/fix problems)

e2fsck -fp /dev/sdX2
resize2fs /dev/sdX2

(Where X is the letter for your thumb drive)

Without a size specified, resize2fs resizes the filesystem to fill the partition.

Hope that helps!

EDIT: I went back and fixed the instructions with this addition now that I've tested it a little more. Unfortunately, I noticed that CWM does not include resize2fs and I didn't spot any other programs handy within CWM to do the trick, so you can't quite do this part on the OUYA. I will look into alternate instructions or building my own CWM that includes resize2fs or just plain automates this more.

~Troop