I took me about 9 minutes to build kernel without "-j". CPU usage for all 8 cores was about 20%.
When I tried "make O=$TEGRA_KERNEL_OUT zImage -j" without "9", CPU usage for all 8 cores was solid 100%, memory usage increased to 100% (16GB) and my HP Z-book locked up.
When I tried "make O=$TEGRA_KERNEL_OUT zImage -j9", CPU usage for all 8 cores was near 100%, memory usage was less than 1GB and build finished in 2 minutes without error.
(Edit: It seems that having 4500 CPU cores (and perhaps even more) could actually be useful (for a short time at least) to compile the Linux kernel ^^)