In order to gain access to program data (not applicable to sdcard), you still need to be either root, or to possess the userid of the particular program whose data you're trying to gain access to. Use of one of those secure-root password prompt programs will give you the ability to limit root access since the 'su' command will fail without the password being entered in the GUI.
This is not absolute though, since you can still boot on a recovery image, backup, and extract. Without actually encrypting the storage, there is no way to absolutely protect your data, and with a mobile device, the encryption/decryption overhead will take up too much CPU time to be practical. It could, however, be implemented on a program-by-program basis or on a data-but-not-program basis, i.e. encrypt /data/data, or /data/data-enc might be a better idea - leave data for user-programs encrypted, but system-services unencrypted, and mount the encrypted partition on screen unlock (i.e. password unlock). LUKS would be great for this. Allowing optional encryption for SD-card and allowing multiple SD-card partitions to be mounted (i.e. one encrypted, one not) would be ideal.