Luckily its still a new phone, so there isn't much on it.
The phone is rooted in that I cannot access it in any way. Even connecting to a computer is useless as its only on for a few seconds before it reboots itself.
CWM recovery isn't installed, will be when the phone is working again though!

Is there a way to wipe the cache/dalvik cache or factory reset while the phone is off?