Here's a more in-depth explanation to 32-bit vs 64-bit.
32-bit has, well, 32 bits for the word length*1. 64-bit has 64 bits for it. The length of a word can be shared across processors (Ex. ARM and x86 both use 32-bit words, IA64 and x86-64 both use 64-bit words)
That's important because a word is the largest value that a CPU can use for things like memory access.
32-bit: 1111 1111 1111 1111 1111 1111 1111 1111
64-bit: 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111
32-bit maximum value: 4,294,967,295d, or 0xFFFFFFFF, or 4 Gigabytes of allocation (This is why a 32-bit system can only allocate a maximum of 4GB of RAM)
64-bit maximum value: 18,446,744,073,709,551,615 or 0xFFFFFFFFFFFFFFFF, or 16 Exabytes of allocation *2
The bitness (32-bit Vs. 64-bit) does not determine the processor architecture. ARM, for example, is traditionally 32-bit (I believe that there is a 64-bit ARM, but I've never seen it in the wild), whereas most modern Intel/AMD CPUs are 64-bit, using the AMD64 instruction set.
THUMB2*3, which is what Windows RT uses, is an extension on the THUMB architecture. THUMB was originally 16-bit, and THUMB2 added 32-bit extensions to it. A lot of what THUMB2 does is only 16-bit (mov/movs for example move 16 bytes into the upper and lower halfwords for a certain register; the MOV32 instruction in MS's ARM assembler is simply a macro that does a mov then a movs), but it does have full support for 32-bit. Opcodes in THUMB2 mode are also a fixed length, unlike ARM.
1: PAE increases the length of a word to 36 bits on older CPUs that are still traditionally considered 32-bit processors.
2: 16 Exabytes of /theoretical/ storage, current hardware doesn't really allow for that. I also want to say that x86-64 has a limitation somewhere around 4TB, but I could be wrong.
3: THUMB/THUMB2 traditionally accompany the ARM instruction sets. The Tegra 3 supports both ARM and THUMB2 hardware wise, but due to THUMB2 being smaller (on account if it only having 16-bit opcodes) with a negligible speed hit MS chose to make WIndows RT only handle THUMB2. They also include what I believe to be a poorly written interrupt handler that always returns back in THUMB2 mode, so, while it's possible to run ARM code the CPU will switch back to THUMB2 at random, as far as your program is concerned.